More Related Content Similar to Github Projects Overview and IBM Streams V4.1 (20) 1. © 2015 IBM Corporation
Github Projects Overview
IBM Streams 4.1
Samantha Chan
IBM Streams Community Architect
For questions about this presentation contact: chanskw@ca.ibm.com
2. 2 © 2015 IBM Corporation
Important Disclaimer
THE INFORMATION CONTAINED IN THIS PRESENTATION IS PROVIDED FOR INFORMATIONAL
PURPOSES ONLY.
WHILE EFFORTS WERE MADE TO VERIFY THE COMPLETENESS AND ACCURACY OF THE
INFORMATION CONTAINED IN THIS PRESENTATION, IT IS PROVIDED “AS IS”, WITHOUT WARRANTY
OF ANY KIND, EXPRESS OR IMPLIED.
IN ADDITION, THIS INFORMATION IS BASED ON IBM’S CURRENT PRODUCT PLANS AND STRATEGY,
WHICH ARE SUBJECT TO CHANGE BY IBM WITHOUT NOTICE.
IBM SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES ARISING OUT OF THE USE OF, OR
OTHERWISE RELATED TO, THIS PRESENTATION OR ANY OTHER DOCUMENTATION.
NOTHING CONTAINED IN THIS PRESENTATION IS INTENDED TO, OR SHALL HAVE THE EFFECT OF:
• CREATING ANY WARRANTY OR REPRESENTATION FROM IBM (OR ITS AFFILIATES OR ITS OR
THEIR SUPPLIERS AND/OR LICENSORS); OR
• ALTERING THE TERMS AND CONDITIONS OF THE APPLICABLE LICENSE AGREEMENT
GOVERNING THE USE OF IBM SOFTWARE.
IBM’s statements regarding its plans, directions, and intent are subject to change or
withdrawal without notice at IBM’s sole discretion. Information regarding potential
future products is intended to outline our general product direction and it should not
be relied on in making a purchasing decision. The information mentioned regarding
potential future products is not a commitment, promise, or legal obligation to deliver
any material, code or functionality. Information about potential future products may
not be incorporated into any contract. The development, release, and timing of any
future features or functionality described for our products remains at our sole
discretion.
THIS INFORMATION IS BASED ON IBM’S CURRENT PRODUCT PLANS AND STRATEGY, WHICH ARE SUBJECT TO CHANGE BY IBM WITHOUT NOTICE.
IBM SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES ARISING OUT OF THE USE OF, OR OTHERWISE RELATED TO, THIS PRESENTATION OR ANY OTHER DOCUMENTATION.
3. 3 © 2015 IBM Corporation
Agenda
IBMStreams Organization
What’s new in Streams Github Projects?
Getting Started with Streams 4.1
4. 4 © 2015 IBM Corporation
IBMStreams
http://ibmstreams.github.io
Open-source organization established on Github in March 2014
Goals:
– Provide a platform and foster a community to extend and share Streams
programming resources (toolkits, samples, performance benchmark,
utilities, etc.)
– Allow us to deliver new toolkit functions in a more open, agile and rapid
manner
– Improve visibility of Streams programming resources and make them more
easily accessible.
5. 5 © 2015 IBM Corporation
IBMStreams
March 2014
– Started the 3 repositories:
• 3 toolkits from the Streams product (HDFS,
Messaging, Inet)
April 2015
– Total: 31 Projects
Dec 2015
– 1 Language Integration
– 9 Adapters
– 6 Parsers and Formatters
– 12 Processing and Analytics
– 6 Utilities
– 8 Demos / Samples / Docs
– Total: 42 Projects
6. 6 © 2015 IBM Corporation
What’s New? – Language Integration
Topology Project (streamsx.topology)
– Shipped as part of Streams v4.1
– A project that supports building streaming topologies (applications) for
Streams in different programming languages, such as Java and Scala.
– An experimental version of Python is available.
7. 7 © 2015 IBM Corporation
What’s New? - Adapters
HBase Toolkit (streamsx.hbase)
– Bluemix Support
HDFS Toolkit (streamsx.hdfs)
– Sequence File Support (Preview)
– Bluemix Support
Messaging Toolkit (streamsx.messaging)
– RabbitMQ (Preview)
– Kafka 0.9
– MessageHub
Inet Toolkit (streamsx.inet)
– The Inet toolkit provides support for common internet protocols. Supported protocols
include FTP, WebSocket, HTTP.
– Native Functions for httpPut, httpGet, httpPost, httpDelete
– Map visualization
8. 8 © 2015 IBM Corporation
What’s New? - Adapters
Multi-Connection TCP Server Toolkit (streamsx.tcp)
– This toolkit contains a TCPServer operator which allows for multi-
connections, and is a multi-threaded source operator. The operator accepts
and sends text or binary data from one or more TCP sockets.
– Added support for full duplex TCP communication. Operator can now
accept and send data via a TCP socket connection.
9. 9 © 2015 IBM Corporation
What’s New? - Adapters
New repositories:
CDC Toolkit (streamsx.cdc)
– Provides support for efficiently reading and writing data from InfoSphere
Change Data Capture (CDC)
GraphDB (streamsx.graphdb)
– Under Construction
– Repository created to provide support for reading and writing data to graph
databases (e.g. IBM System G / Apache Tinkerpop)
JDBC (streamsx.jdbc)
– JDBCRun operator to allow IBM Streams application to run queries from DB
via JDBC
10. 10 © 2015 IBM Corporation
What’s New? – Parsers and Formatters
Parquet Toolkit (streamsx.parquet)
–Parquet is a columnar storage format for Hadoop. This repository is created for
hosting operators for reading and writing data in Parquet format.
11. 11 © 2015 IBM Corporation
What’s New? – Analytics and Processing
SparkMLLib (streamsx.sparkmllib)
– This repository contains a toolkit for real-time scoring using SparkMLLib.
Healthcare Toolkit (streamsx.health)
– This repository is under construction. This toolkit is intended to provide
building blocks and operators for real-time healthcare related analysis.
Social Toolkit (streamsx.social)
– This repository provides support to ingest data from social media. Initial
support includes ingesting from Twitter
12. 12 © 2015 IBM Corporation
What’s New? – Analytics and Processing
Anomaly Detection (streamsx.anomalyDetection)
– This repository contains operators and functions for anomaly detection. Initial
contribution would be based on the Local Outlier Function (LOF)
Internet of Things Foundation Toolkit (streamsx.iotf)
– Provides the ability to have IBM Streams application to easily interact with
IoTF, either in Bluemix or on-premises.
13. 13 © 2015 IBM Corporation
Github Projects Overview – Utilities
Network Toolkit (streamsx.network)
– Contains operators and functions for processing network data.
Transform Toolkit (streamsx.transform)
– Contains building block operators to transform input data from one format to
another.
– Contains a high-performance Modify operator that can be used as a
replacement for Functor
14. 14 © 2015 IBM Corporation
Github Projects Overview – Samples / Demos / Docs
Cybersecurity Starter Applications
(streamsx.cybersecurity.starterApps)
– Contains starter applications to get started with the IBM Streams
Cybersecurity Toolkit quickly
Tutorials
– Contains Streams Introduction and SPSS labs
Streamsx.documentation
– Community-based documentation
– Contains various getting started guides for Streams v4.1 features
15. 15 © 2015 IBM Corporation
Getting Started! - StreamsDev
https://developer.ibm.com/streamsdev/
16. 16 © 2015 IBM Corporation
Streamsx.documentation
Community-based documentation project
Complements StreamsDev and Knowledge Center
Provides Getting Started Guides for new features in Streams
Community contribution and feedback is much appreciated!
http://ibmstreams.github.io/streamsx.documentation/
17. 17 © 2015 IBM Corporation
Get Involved!
Many new and cool projects to try!
– E.g. Python Support, JDBC, Social
We want your feedback and input!
– Report any issues you have found
– If you think it’s cool, let us know!
– Which toolkit do you use? How do you use it?
– Do you want the toolkit to be included in the product?
Contribute code, samples and docs!
– Got a clever way to do things? Contribute it to the samples project!
– Need a new parameter? Need support for a new type of server / data format?
Work with us to try to implement it!
– Get a better way to explain things? Contribute to streamsx.documentation
Got an idea?
– Propose a new project / new feature.
