- Home
- Data Marketplace Software
- data.all
data.all
What it is
data.all is an open source platform deployed in AWS accounts to help teams manage and discover data assets. Its centralized catalog inventories datasets, tables, folders, and dashboards, with filters for asset type, tags, region, and glossary terms. Datasets can hold structured tables or unstructured folders and use AWS S3, KMS, IAM, and Glue resources. Teams can share datasets across environments by granting selected roles read access through Lake Formation without moving the data. The catalog can import tables from Redshift Serverless and provisioned clusters. Analytics and machine learning options include Athena worksheets, Jupyter notebooks and SageMaker Studio access, and QuickSight dashboards. Administrators can organize teams and users across environments. Authentication can use Cognito federated with a corporate identity provider or an external OIDC provider, with group information supplied by the identity provider. Deployments can face the internet or remain within a VPC, where the UI and APIs are reachable only inside it. AWS deployment requires at least two accounts in matching regions and administrator credentials for both the tooling and deployment accounts. A Docker Compose local setup is also documented, and AWS credentials are needed to create environments and datasets.
For and against
For
- Catalog covers datasets, tables, folders, and dashboards.
- Lake Formation sharing grants selected roles read access.
- Supports Redshift table imports.
- Offers Athena, SageMaker, Jupyter, and QuickSight options.
- Internet-facing and VPC-facing deployments are described.
Against
- AWS deployment requires at least two accounts.
- The tooling and deployment accounts must use matching regions.
- Administrator credentials are required during setup.
- AWS credentials are needed to create local environments and datasets.
The verdict
data.all combines data discovery, cataloging, sharing, and analytics options in an open source platform for AWS environments. Its account, region, and credential requirements are important deployment considerations.
Who it suits
data.all may suit teams managing data assets in AWS accounts, including those seeking shared catalogs, controlled dataset access, and analytics or machine learning tools. Deployment requires AWS account setup and administrator credentials.
On the record
- Administration
- Administrators can organize and manage teams and users across environments.data-dot-all.github.io · read 8 Oct 2026
- Analytics and ML
- The platform provides Athena worksheets, Jupyter notebooks through SageMaker, SageMaker Studio access, and QuickSight dashboards.data-dot-all.github.io · read 8 Oct 2026
- Audit
- For structured data, SQL queries from EMR, Redshift, Glue Jobs, and Athena are captured through Lake Formation.data-dot-all.github.io · read 8 Oct 2026
- AWS integrations
- The documented architecture uses AWS services including Cognito, CloudFront or an internal Application Load Balancer, API Gateway, Lambda, Aurora PostgreSQL, ECS Fargate, and OpenSearch.data-dot-all.github.io · read 8 Oct 2026
- Catalog
- Its centralized catalog inventories datasets, tables, folders, and dashboards, with search filters for asset type, tags, region, and glossary terms.data-dot-all.github.io · read 8 Oct 2026
- Configuration
- Modules and some features can be enabled or disabled through configuration, and the guide notes that file uploads are disabled in its example configuration.data-dot-all.github.io · read 8 Oct 2026
- Data catalog
- Its centralized catalog inventories datasets, tables, folders, and dashboards, with search filters for asset type, tags, region, and glossary terms.data-dot-all.github.io · read 8 Oct 2026
- Datasets
- Datasets can hold structured tables or unstructured folders and use AWS S3, KMS, IAM, and Glue resources.data-dot-all.github.io · read 8 Oct 2026
- Data sharing
- Users can request access to another team’s data assets, and asset owners can accept or reject those requests.data-dot-all.github.io · read 8 Oct 2026
- Deployment
- The AWS deployment guide requires a tooling account and at least one deployment account, and their selected regions must match.data-dot-all.github.io · read 8 Oct 2026
- Deployment options
- The deployment guide describes internet-facing and VPC-facing architectures; the VPC-facing option makes the UI and APIs reachable only within the VPC.data-dot-all.github.io · read 8 Oct 2026
- Identity
- Authentication can use Amazon Cognito federated with a corporate identity provider, or an external OIDC provider; data.all relies on identity-provider group information.data-dot-all.github.io · read 8 Oct 2026
- Integrations
- Documented modules include AWS Glue, Lake Formation, Amazon S3, Athena, SageMaker, QuickSight, and HealthOmics Ready2Run workflows.data-dot-all.github.io · read 8 Oct 2026
- Local use
- The project documents a Docker Compose local setup with frontend, database, GraphQL, CDK proxy, and OpenSearch containers; AWS credentials are needed to create environments and datasets.data-dot-all.github.io · read 8 Oct 2026
- Network options
- Deployments can be internet facing, with public APIs protected by AWS WAF, or VPC facing, with private sites and APIs accessible through VPC endpoints.data-dot-all.github.io · read 8 Oct 2026
- Notebooks
- The platform provides notebooks based on Amazon SageMaker Studio for data scientists to build machine learning models.data-dot-all.github.io · read 8 Oct 2026
- Operational limit
- The AWS deployment instructions require administrator credentials for the tooling and deployment accounts during setup.data-dot-all.github.io · read 8 Oct 2026
- Purpose
- data.all is an open source platform that can be deployed in AWS accounts to help users manage and discover data assets.data-dot-all.github.io · read 8 Oct 2026
