Agentic AI Atlasby a5c.ai
OverviewWikiGraphFor AgentsEdgesSearchWorkspace
/
GitHubDocsDiscord
iiRecord
Agentic AI Atlas · Data Engineer
role:data-engineera5c.ai
Search record views/
Record · tabs

Available views

II.Record viewspp. 1 - 1
overviewjsongraph
II.
Role overview

role:data-engineer

Reference · live

Data Engineer overview

Designs and operates the data infrastructure — pipelines, warehouses, and streaming systems — that move raw data from source systems to analytics and product consumers in a reliable, scalable, and governed way. Partners with data scientists, analysts, and product teams to understand data needs and model datasets appropriately. Owns data quality monitoring and pipeline observability.

RoleOutgoing · 18Incoming · 107

Attributes

displayName
Data Engineer
isAgentic
false
automatability
0.8
description
Designs and operates the data infrastructure — pipelines, warehouses, and streaming systems — that move raw data from source systems to analytics and product consumers in a reliable, scalable, and governed way. Partners with data scientists, analysts, and product teams to understand data needs and model datasets appropriately. Owns data quality monitoring and pipeline observability.
seniority
mid

Outgoing edges

holds_responsibility7
  • responsibility:data-quality·ResponsibilityData Quality
  • responsibility:on-call·ResponsibilityOn-Call
  • responsibility:documentation·ResponsibilityDocumentation
  • responsibility:performance-optimization·ResponsibilityPerformance Optimization
  • responsibility:data-quality·ResponsibilityData Quality
  • responsibility:data-quality-monitoring·ResponsibilityData quality monitoring
  • responsibility:documentation·ResponsibilityDocumentation
requires_expertise7
  • skill-area:etl-pipelines·SkillAreaETL Pipelines
  • skill-area:data-warehouse-modeling·SkillAreaData Warehouse Modeling
  • skill-area:dbt-modeling·SkillAreadbt Modeling
  • skill-area:data-quality·SkillAreaData Quality
  • skill-area:data-lineage·SkillAreaData Lineage
  • skill-area:spark-jobs·SkillAreaApache Spark Jobs
  • skill-area:batch-vs-stream-tradeoffs·SkillAreaBatch vs Stream Tradeoffs
requires_skill4
  • domain:data-engineering·DomainData Engineering
  • specialization:data-engineering-analytics·Specialization
  • domain:databases·DomainDatabases
  • domain:cloud-infra·DomainCloud Infrastructure

Incoming edges

has_member3
  • org-unit:data-team·OrgUnitData Team
  • org-unit:data-platform-team·OrgUnitData Platform Team
  • org-unit:business-intelligence-team·OrgUnitBusiness Intelligence Team
held_by2
  • responsibility:data-quality-monitoring·ResponsibilityData quality monitoring
  • responsibility:data-pipeline-reliability·ResponsibilityData pipeline reliability
involves_role17
  • workflow:data-pipeline-deployment·WorkflowData Pipeline Deployment
  • workflow:ml-model-lifecycle·WorkflowML Model Lifecycle
  • workflow:data-quality-monitoring·WorkflowData Quality Monitoring
  • workflow:data-governance-review·WorkflowData Governance Review
  • workflow:schema-migration·WorkflowSchema Migration
  • workflow:data-backfill-procedure·WorkflowData Backfill Procedure
  • workflow:synthetic-data-generation-pipeline·WorkflowSynthetic Data Generation Pipeline
  • workflow:data-quality-scorecard-review·WorkflowData Quality Scorecard Review
  • workflow:analytics-instrumentation-audit·WorkflowAnalytics Instrumentation Audit
  • workflow:self-serve-analytics-enablement·WorkflowSelf-Serve Analytics Enablement
  • workflow:enterprise-data-platform-health-check·WorkflowEnterprise Data Platform Health Check
  • workflow:data-quality-investigation·WorkflowData Quality Investigation
  • workflow:model-training-pipeline·WorkflowModel Training Pipeline
  • workflow:data-quality-check·WorkflowData Quality Check
  • workflow:data-migration·WorkflowData Migration
  • workflow:alternative-data-evaluation·WorkflowAlternative Data Evaluation
  • workflow:market-data-feed-validation·WorkflowMarket Data Feed Validation
lib_involves_role53
  • lib-agent:code-migration-modernization--data-architect-agent·LibraryAgentdata-architect-agent
  • lib-agent:code-migration-modernization--data-integrity-validator·LibraryAgentdata-integrity-validator
  • lib-agent:code-migration-modernization--database-migration-orchestrator·LibraryAgentdatabase-migration-orchestrator
  • lib-agent:data-engineering-analytics--data-governance-steward·LibraryAgentData Governance Steward Agent
  • lib-agent:data-engineering-analytics--data-orchestration-engineer·LibraryAgentData Orchestration Engineer Agent
  • lib-agent:data-engineering-analytics--data-quality-engineer·LibraryAgentdata-quality-engineer
  • lib-agent:data-engineering-analytics--data-warehouse-architect·LibraryAgentdata-warehouse-architect
  • lib-agent:data-engineering-analytics--dbt-project-engineer·LibraryAgentdbt-project-engineer
  • lib-agent:data-engineering-analytics--migration-specialist·LibraryAgentMigration Specialist Agent
  • lib-agent:data-engineering-analytics--ml-feature-engineer·LibraryAgentML Feature Engineer Agent
  • lib-agent:data-engineering-analytics--streaming-pipeline-engineer·LibraryAgentStreaming Pipeline Engineer Agent
  • lib-agent:data-science-ml--data-engineer·LibraryAgentdata-engineer
  • lib-agent:data-science-ml--feature-store-engineer·LibraryAgentfeature-store-engineer
  • lib-agent:knowledge-management--kg-specialist·LibraryAgentkg-specialist
  • lib-agent:travel--python-etl-engineer·LibraryAgentpython-etl-engineer
  • lib-agent:travel--sql-query-composer·LibraryAgentsql-query-composer
  • lib-agent:travel--sqlite-schema-architect·LibraryAgentsqlite-schema-architect
  • lib-agent:physics--research-data-manager·LibraryAgentresearch-data-manager
  • lib-agent:healthcare--clinical-informatics-specialist·LibraryAgentclinical-informatics-specialist
  • lib-agent:software-architecture--data-architect·LibraryAgentdata-architect
  • lib-process:software-architecture--data-architecture-design·LibraryProcessspecializations/software-architecture/data-architecture-design
  • lib-skill:code-migration-modernization--data-migration-validator·LibrarySkilldata-migration-validator
  • lib-skill:code-migration-modernization--etl-pipeline-builder·LibrarySkilletl-pipeline-builder
  • lib-skill:code-migration-modernization--query-translator·LibrarySkillquery-translator
  • lib-skill:code-migration-modernization--schema-comparator·LibrarySkillschema-comparator
  • lib-skill:common-utilities--json-data-wrangling·LibrarySkilljson-data-wrangling
  • lib-skill:cryptography-blockchain--subgraph-indexing·LibrarySkillsubgraph-indexing
  • lib-skill:data-engineering-analytics--airflow-dag-analyzer·LibrarySkillairflow-dag-analyzer
  • lib-skill:data-engineering-analytics--apache-spark-optimizer·LibrarySkillApache Spark Optimizer
  • lib-skill:data-engineering-analytics--batch-vs-stream-tradeoffs·LibrarySkillbatch-vs-stream-tradeoffs
  • lib-skill:data-engineering-analytics--cdc-pattern-implementer·LibrarySkillCDC Pattern Implementer
  • lib-skill:data-engineering-analytics--cost-optimizer·LibrarySkillCost Optimizer (Cloud Data Platforms)
  • lib-skill:data-engineering-analytics--data-catalog-enricher·LibrarySkillData Catalog Enricher
  • lib-skill:data-engineering-analytics--data-lineage-mapper·LibrarySkilldata-lineage-mapper
  • lib-skill:data-engineering-analytics--data-quality-profiler·LibrarySkilldata-quality-profiler
  • lib-skill:data-engineering-analytics--dbt-project-analyzer·LibrarySkilldbt-project-analyzer
  • lib-skill:data-engineering-analytics--dimensional-model-validator·LibrarySkillDimensional Model Validator
  • lib-skill:data-engineering-analytics--etl-testing·LibrarySkilletl-testing
  • lib-skill:data-engineering-analytics--feature-engineering-optimizer·LibrarySkillFeature Engineering Optimizer
  • lib-skill:data-engineering-analytics--great-expectations-generator·LibrarySkillGreat Expectations Generator
  • lib-skill:data-engineering-analytics--incremental-model-strategy-selector·LibrarySkillIncremental Model Strategy Selector
  • lib-skill:data-engineering-analytics--kafka-topic-designer·LibrarySkillKafka Topic Designer
  • lib-skill:data-engineering-analytics--obt-design-optimizer·LibrarySkillOBT Design Optimizer
  • lib-skill:data-engineering-analytics--scd-implementation-generator·LibrarySkillSCD Implementation Generator
  • lib-skill:data-engineering-analytics--schema-evolution-manager·LibrarySkillSchema Evolution Manager
  • lib-skill:data-engineering-analytics--spark-jobs·LibrarySkillspark-jobs
  • lib-skill:data-engineering-analytics--sql-query-optimizer·LibrarySkillsql-query-optimizer
  • lib-skill:data-engineering-analytics--stream-processing-windowing-designer·LibrarySkillStream Processing Windowing Designer
  • lib-skill:data-science-ml--feast-feature-store·LibrarySkillfeast-feature-store
  • lib-skill:knowledge-management--autocomplete-engine·LibrarySkillautocomplete-engine
  • lib-skill:healthcare--health-data-integration·LibrarySkillhealth-data-integration
  • lib-skill:humanities--metadata-standards-implementation·LibrarySkillmetadata-standards-implementation
  • lib-skill:humanities--tei-text-encoding·LibrarySkilltei-text-encoding
supports_work4
  • tool:databricks·ToolDatabricks
  • tool:snowflake·ToolSnowflake
  • tool:dbt-labs·Tooldbt Labs
  • tool-server:mcp-dbt-candidate·ToolServerdbt MCP candidate
used_by_role28
  • stack-profile:data-lakehouse·StackProfileData Lakehouse Stack (Databricks, Spark, Delta Lake, dbt, Airflow)
  • stack-profile:stream-processing·StackProfileStream Processing Stack (Kafka, Flink, Schema Registry, Prometheus)
  • stack-profile:feature-store-mlops·StackProfileFeature Store & MLOps Stack (Feast, MLflow, BentoML, K8s, Prometheus)
  • stack-profile:edge-ai-iot·StackProfileEdge AI / IoT Stack (TensorFlow Lite, MQTT, Rust, InfluxDB, Grafana)
  • stack-profile:search-discovery·StackProfileSearch & Discovery (Elasticsearch/Meilisearch + Python/Node.js + Redis)
  • stack-profile:document-processing-pipeline·StackProfileDocument Processing Pipeline (OCR + NLP + Python + Elasticsearch + FastAPI)
  • stack-profile:geospatial-analytics·StackProfileGeospatial Analytics (PostGIS + Python + Leaflet/Mapbox + GeoPandas)
  • stack-profile:batch-processing·StackProfileBatch Processing (Airflow + dbt + PostgreSQL + Python + S3)
  • stack-profile:insurance-claims-processing·StackProfileInsurance Claims Processing Stack (Python, FastAPI, PostgreSQL, Kafka, OCR/ML)
  • stack-profile:adtech-real-time-bidding·StackProfileAdTech Real-Time Bidding Stack (Go, Redis, Kafka, ClickHouse, Prometheus)
  • stack-profile:knowledge-graph-platform·StackProfileKnowledge Graph Platform (Neo4j, Python, FastAPI, React, D3, Elasticsearch)
  • stack-profile:synthetic-data-generation·StackProfileSynthetic Data Generation Stack (Python, PyTorch, FastAPI, PostgreSQL, S3)
  • stack-profile:data-warehouse-bi·StackProfileData Warehouse / BI Stack (dbt, BigQuery, Metabase/Looker, Python, Airflow)
  • stack-profile:time-series-analytics·StackProfileTime-Series Analytics Stack (InfluxDB, Grafana, Telegraf, Python, Go)
  • stack-profile:data-quality-governance·StackProfileData Quality / Governance Stack (Great Expectations, dbt, Airflow, PostgreSQL, Python)
  • stack-profile:enterprise-search-platform·StackProfileEnterprise Search (Elasticsearch, Python, React, Redis, Kubernetes, FastAPI)
  • stack-profile:master-data-management·StackProfileMaster Data Management (Python, PostgreSQL, RabbitMQ, Airflow, FastAPI)
  • stack-profile:research-data-platform·StackProfileResearch Data Platform (Python, Jupyter, PostgreSQL, Boto3, FastAPI, React)
  • stack-profile:social-network-backend·StackProfileSocial Network Backend (Go, PostgreSQL, Redis, RabbitMQ, Elasticsearch, Boto3)
  • stack-profile:julia-data-service·StackProfileJulia Data Service (Julia, Python, PostgreSQL, Docker)
  • stack-profile:cqrs-read-model·StackProfileCQRS Read Model (TypeScript, Elasticsearch, Kafka, PostgreSQL, NestJS, Docker)
  • stack-profile:data-pipeline-orchestration·StackProfileData Pipeline Orchestration (Python, Airflow, dbt, PostgreSQL, Docker)
  • stack-profile:etl-reverse-etl·StackProfileETL / Reverse ETL (Python, Airbyte, dbt, PostgreSQL, Airflow)
  • stack-profile:real-time-analytics-stack·StackProfileReal-Time Analytics Stack (Kafka, ClickHouse, Grafana, dbt)
  • stack-profile:fastapi-postgres·StackProfileFastAPI + SQLAlchemy + PostgreSQL
  • stack-profile:rag-stack·StackProfileRAG Application Stack (Python + LangChain + ChromaDB + FastAPI)
  • stack-profile:data-lake-stack·StackProfileData Lake Stack (Spark, Object Storage, Delta/Iceberg)
  • stack-profile:event-driven-stack·StackProfileEvent-Driven Stack (Kafka, Consumers, Schema Registry)

Related pages

No related wiki pages for this record.

Shortcuts

Open in graph
Browse node kind