Tom Sawyer Data Streams, Web Edition

Release 1.1.0

Overview

Tom Sawyer Data Streams 1.1.0 features direct integration with IBM Db2, Microsoft SQL Server, MySQL, Oracle, and PostgreSQL database sources. This release also features the ability to inspect the schema of individual source, condition, transformation, and sink nodes in data streams. Several updates improve the performance of data streams in high-throughput cases.

Tom Sawyer Data Streams has been updated to use Tom Sawyer Perspectives 13.7.0. The new ortholinear diagramming behavior preserves orthogonality of edges on drop actions in order to provide cleaner handling of dynamic interactions and more stable and visually consistent diagrams. You can see more details about Tom Sawyer Perspectives 13.7.0 here. Also, many third-party libraries have been updated to eliminate potential vulnerabilities.

New

Data Integration

• Data integration from IBM Db2, Microsoft SQL Server, MySQL, Oracle, and PostgreSQL database sources

Schema View

• Show input schema on sink, condition, and transformation nodes in data streams
• Show output schema on condition, transformation, and source nodes in data streams

Improvements

Performance

• Made several updates to improve performance of string and hash map calculations in high throughput cases

Data Stream

• Automatic filtering of element type options in dialogs to consider previous sources, conditions, and transformations
• Made list of suggestions searchable to facilitate faster building of data streams
• Added suggestion button to Condition nodes to facilitate faster building of data streams

System Requirements

• Added support for the following operating systems:
• Apple macOS 27
• Ubuntu Linux 26.04 LTS
• Added support for the following data sources:
• IBM Db2 12.1
• Kafka 8.3
• Microsoft SQL Server 17.0
• MySQL 26.7
• Oracle 26ai
• PostgreSQL 18.6
• Added support for Docker Engine 29.7
• Added support for Keycloak 26.7
• Added support for the Brave web browser
• Dropped support for the following operating systems:
• Apple macOS 15
• Ubuntu Linux 24.04 LTS
• Dropped support for the following data sources:
• Kafka 8.1
• PostgreSQL 17
• Dropped support for Docker Engine 29.1
• Dropped support for Keycloak 26.0

Release 1.0.0

Overview

Tom Sawyer Data Streams introduces a powerful new approach to Extract Transform Load (ETL) and unification of streaming and legacy data into a single, governed, query-ready knowledge graph in real time. The platform seamlessly subscribes to Apache Kafka (or Confluent) topics—sourced from relational databases, data warehouses, unstructured data, APIs, and more—allowing you to consolidate disparate data without expensive migrations. This release supports Kafka authentication and encryption along with OAuth 2.0 and Keycloak. Deployment is streamlined across cloud, on-premises, and air-gapped environments using a dedicated Docker installer.

The core of the release emphasizes agility and precision in data preparation. An intuitive, web-based designer enables visual data flow design, allowing users to define sources and sinks, and apply powerful transformations without writing custom code. Data flows can run continuously, providing real-time, low-latency processing that keeps the knowledge graph perpetually current for operational decisions and analytics. The resulting graph can be persisted in a graph database and published back to Kafka, supporting downstream tools like AI, visualization, and retrieval-augmented generation (RAG).

Data Streams pairs directly with Tom Sawyer Perspectives and Tom Sawyer Explorations to instantly visualize the resulting knowledge graph. This direct link moves you from data ingestion to interactive insight in minutes, providing automatic graph layouts, powerful filtering, and intuitive data exploration, enabling all stakeholders to validate models and share critical findings. Data Streams has been built with Tom Sawyer Perspectives 13.4.0.

New

Data Integration

• Data integration from Kafka topics
• Data integration from PostgreSQL Debezium sources
• Save models to JanusGraph databases
• Save models to Kafka topics
• Real-time CRUD support

Schema View

• Automatic schema extraction for Kafka element types, attribute names, and data types
• Support for Kafka sampling
• Support for Confluent Schema Registry

Data Flow

• Visual environment to define data sources, transformations, conditions, and sinks
• Automatic graph layout of data flows
• Duplication and renaming of data flows
• Data flow validation
• Editing of data flows
• Synchronized with schema view for immediate reflection of data source changes and schema transformations
• Transformations
  • Convert from node type to edge type
  • Join
  • Rename element
  • Set attribute
  • Set ID
  • Split
  • Support for Spring Expression Language (SpEL)
• Batch processing
• Real-time, continuous processing

Authentication

• OAuth 2.0 authentication
• Keycloak authentication

Installer

• Docker-based installer
• Support for airgapped environments

Documentation

• Setup Guide
• User Guide