at Financial Technnology Year
Please contact them if you have any questions.
Features in Investment Data Warehouses
for Data Management
Centralized repositories designed to store, organize, and make accessible various types of investment data including historical prices, positions, transactions, and analytical datasets.
We ask vendors to mark their product against our feature list. It is not available for all products and in some cases we have derived data from publicly available sources. The list of features follows. Vendors can add a further 10 key features of their own.
Analytical & Reporting Tools
Features supporting analysis, reporting, and insights from stored investment data.
- API for Data Export
- APIs for exporting data to downstream analytics/BI tools.
- Ad-Hoc Query Tools
- User ability to run custom queries without IT intervention.
- Audit Reports
- Automated generation of compliance or data access audit reports.
- Data Visualization
- Support for charting, heatmaps, and other visualization outputs.
- Integrated Reporting
- Provision of built-in or plug-in dashboards and report templates.
- Natural Language Query
- Supports querying data using plain English or natural language.
- OLAP (Online Analytical Processing)
- Multi-dimensional data cubes for advanced slicing/dicing.
- Scheduled Reporting
- Capability to schedule and distribute recurring reports.
- Self-Service BI Integration
- Connection with leading business intelligence tools (Power BI, Tableau, etc).
- User-Defined Metrics & Calculations
- Ability for users to create custom metrics and run calculations.
Data Governance & Compliance
Controls and processes to ensure regulatory compliance, auditability, and best governance practices.
- Automated Regulatory Reporting
- Template-driven and automated generation of regulatory filings.
- Change Management & Approval Workflow
- Approval process tracking for critical data or schema changes.
- Consent Management
- Tools to record and enforce user/client consent for data use.
- Data Retention Policies
- Automated enforcement of data lifespan per regulatory requirements.
- Data Stewardship Assignment
- Assign roles for stewardship and ongoing data oversight.
- Data Usage Monitoring
- Track and report data access frequency and patterns.
- E-discovery Support
- Tools for responding to legal or regulatory data queries.
- Full Audit Trail
- Detailed, immutable logging of data access and changes for compliance.
- GDPR/CCPA Compliance Modules
- Functions for supporting global privacy laws in data management.
- Policy Documentation Portal
- Online repository for governance, access, and retention policies.
Data Integration & Ingestion
Capabilities related to connecting with, importing, and consolidating data from various sources into the warehouse.
- API Access for Ingestion
- Availability of APIs to push/pull data from external systems.
- Auto Schema Detection
- Automated detection and suggestion for new/unknown data schemas.
- Batch Processing Support
- Supports bulk or scheduled data loads at defined intervals.
- Customizable Workflows
- Support for creating custom data ingestion and integration workflows.
- Data Mapping/Transformation
- Ability to map, normalize, or transform source data to internal schemas.
- Data Validation Rules
- Automated quality checks during ingestion for format, completeness, or range validation.
- ETL (Extract, Transform, Load) Tools
- Built-in or integrated ETL processes for complex workflows.
- Error Logging & Alerting
- Automated notifications or logs on ingestion failures, anomalies, or discrepancies.
- Latency
- Time taken from data arrival at source to availability in warehouse.
- Multi-Source Connectivity
- Ability to connect to multiple data sources such as custodians, brokers, market data feeds, CRM, and internal systems.
- Real-Time Ingestion
- Support for real-time or near real-time data ingestion and updates.
- Scalability
- Maximum capacity of records the ingestion process can handle per hour.
- Streaming Data Capability
- Ingestion and management of continuously flowing data (e.g., tick data, news feeds).
Data Quality & Management
Tools and features for ensuring the accuracy, completeness, and integrity of stored data.
- Automated Data Quality Checks
- Scheduled or real-time scans for inconsistencies, outages, and data drift.
- Custom Validation Rules
- Administrators can define custom data validation and alert logic.
- Data Anomaly Detection
- Automated identification of abnormal or suspicious data patterns.
- Data Deduplication
- Automated identification and resolution of duplicate records.
- Data Lineage Tracking
- Full traceability of data sources and transformations.
- Data Reconciliation Tools
- Tools for aligning positions, transactions, and market values with external sources.
- Error Correction Workflow
- Built-in process for reviewing and resolving flagged issues.
- Historical Data Versioning
- Retains and enables access to previous versions of records.
- Metadata Management
- Cataloging of data with descriptive metadata, tags, and classifications.
- Quality Score Metrics
- Quantitative scoring of data quality or completeness.
Data Security & Access Control
Features safeguarding sensitive investment data and enforcing proper user permissions.
- Audit Logging
- Captures logs of all user actions for compliance and review.
- Certifications/Compliance
- System meets external standards (e.g. SOC 2, ISO 27001, GDPR).
- Data Access Expiry
- Ability to grant time-limited access to specific datasets.
- Data Encryption In Transit
- Secures data as it moves between systems using transport-layer encryption.
- Data Masking
- Conceals sensitive fields from unauthorized users while keeping them available for queries.
- Field Level Encryption
- Ability to encrypt specific columns or fields within the warehouse.
- Granular Permissions
- Allows field-, table-, or dataset-level restrictions on view/edit
- Multi-Factor Authentication (MFA)
- Requires multiple authentication methods for enhanced security.
- Role-Based Access Controls (RBAC)
- Enables fine-grained permissions based on user roles and responsibilities.
- Security Incident Alerting
- Automated alerts triggered by unusual activities or access violations.
- Single-Sign-On (SSO) Integration
- Integration with enterprise authentication solutions for single sign-on.
- User Provisioning Automation
- Streamlines onboarding/offboarding of user access rights.
Data Storage & Architecture
Properties and options related to the physical and logical storage of datasets within the warehouse.
- Cloud/On-Premises Option
- Deployment flexibility on cloud, on-premises or hybrid environments.
- Columnar vs. Row Storage
- Choice between columnar or row-based data storage, or hybrid models.
- Data Compression
- Support for storage-efficient compression algorithms.
- Data Encryption at Rest
- Uses encryption to secure data at rest on storage devices.
- Data Snapshots
- Capability to take periodic snapshots or backups of stored data.
- Elastic Scaling
- Ability to dynamically scale storage resources up/down as needed.
- High Availability & Redundancy
- Built-in redundancy for data protection and system resilience.
- Immutable Data Storage
- Supports 'write-once, read-many' records for archiving and audit trails.
- Object Storage Integration
- Ability to integrate with external object stores (like S3, Azure Blob, etc).
- Partitioning & Sharding
- Support for data partitioning or sharding for optimized performance.
- Schema Evolution
- Capability to modify and evolve storage schemas with minimal disruption.
- Storage Capacity
- Maximum volume of data that can be stored.
Interoperability & Integration
Features that facilitate connectivity and integration with other platforms or applications.
- Bulk Data Transfer Support
- Efficient mechanisms for exporting/importing large datasets.
- Custom Plugin Support
- Ability to extend functionality via custom plugins.
- Data Export Connectors
- Pre-built connectors to downstream systems (accounting, risk, performance).
- Data Import Connectors
- Connectors for ingesting data from industry-standard vendors and records.
- Data Synchronization Scheduling
- Configure schedules and triggers for sync with external applications.
- Event Driven Architecture
- Ability to publish/subscribe to changes via events (e.g., webhooks, Kafka).
- Federated Query Support
- Query data across multiple sources without data movement.
- Integration Testing Sandbox
- Provides a test environment for third-party integration validation.
- Open Standards Adoption
- Uses open-source or de-facto industry schemas and protocols.
- Standardized API Interfaces
- Support for REST, GraphQL, or other industry-standard APIs.
Monitoring, Maintenance & Support
Capabilities around system health monitoring, support, and operational reliability.
- 24/7 Technical Support
- Access to technical support all day, every day.
- Automated Backups
- Scheduled and on-demand backup management.
- Automated Resource Scaling
- Dynamic allocation of compute and storage as workload changes.
- Disaster Recovery Support
- Plans and automation for fast recovery from critical failures.
- Incident Management Dashboard
- Central dashboard for viewing, tracking, and resolving operational incidents.
- Patch & Version Management
- Tools for applying updates and managing software versions.
- Performance Baselines
- Historical records of key performance baselines for benchmarking.
- Real-Time System Monitoring
- Visibility into performance, capacity, and health via dashboards.
- Self-Healing Capabilities
- Automated detection and mitigation of system faults.
- Service Level Agreement (SLA) Uptime
- Guaranteed system availability percentage per SLA.
Performance & Scalability
Measures and features that support rapid queries and a growing investment data landscape.
- Concurrent Users Supported
- Maximum number of simultaneous active users.
- Data Caching
- Layered caching to speed up frequently accessed queries.
- Elastic Compute Scaling
- Ability to add processing resources dynamically as workload grows.
- Geographic Distribution
- Ability to distribute data and resources across multiple locations.
- High Availability
- Uptime/availability percentage supported by system architecture.
- Hybrid Scaling
- Support for both vertical (bigger servers) and horizontal (more servers) growth.
- Max Query Throughput
- Highest number of queries handled per second.
- Query Response Time
- Average time for user queries to complete.
- Resource Utilization Optimizations
- Automated resource allocation and optimization for performance.
- Workload Isolation
- Isolating analytic and operational workloads to prevent interference.
User Experience & Administration
User interface and admin tools to ease operation, management, and adoption.
- Accessibility Compliance
- Adheres to WCAG or other accessibility standards.
- Bulk User Management
- Batch provisioning, editing, or deactivation of users.
- Custom Notification Settings
- Users can configure their own notification preferences.
- Customizable Dashboards
- Personalized dashboards for different user roles.
- In-platform Help & Support
- Contextual help guides, live chat, or ticket escalation.
- Intuitive User Interface
- Modern, easy-to-navigate web UI for end users and admins.
- Localization and Multi-language Support
- UI and documentation in multiple languages.
- System Usage Analytics
- Track and visualize user adoption and active usage trends.
- Theming & Branding Options
- Support for organization-specific visual branding.
- User Activity Monitoring
- Admin panel to review recent logins and actions.