Pipelines Overview

The Pipelines Dashboard provides a bird's-eye view of all the pipelines in your system, their current status, and key metrics. Use this dashboard to monitor pipeline health and navigate to specific pipelines for deeper analysis.

---

Accessing the Pipelines Dashboard

1. Open Datograde and navigate to Pipelines from the left-hand sidebar.
2. The dashboard displays:
   • A list of all pipelines with their current status (e.g., Healthy, Degraded, Failed).
   • Key metrics for each pipeline:
     • Success Rate: Percentage of successful executions.
     • Average Latency: Time taken to process data.
     • Error Rate: Percentage of executions that failed.
     • Throughput: Number of data units processed per minute/hour.

---

Navigating a Specific Pipeline

1. Click on a pipeline from the dashboard to open its Pipeline Map.
2. The Pipeline Map shows:
   • Input Sources: Where data enters the pipeline.
   • Processing Steps: Each transformation or logic stage.
   • Output Destinations: Final outputs of the pipeline.
3. Use the interactive map to:
   • Select specific steps for inspection.
   • View data flow and dependencies.

---

Identifying Problematic Pipelines

1. Use the Filters and Search Bar in the dashboard to locate pipelines based on:
   • Tags (e.g., production, beta, high_priority).
   • Error rates or latency thresholds.
2. Pipelines with issues will be flagged with alerts (e.g., High Error Rate).

---

Next Steps

• Inspecting and Debugging Pipeline Steps
• Monitoring Pipeline Performance