Automated Crypto Data Pipeline
A full-stack data engineering project demonstrating an end-to-end automated ETL pipeline.
Market Analytics
Top 5 Assets Performance (%)
Relative performance rebased to 0% at the start of the period. Comparing XRP, ETH, BNB, USDT, BTC.
Market Capitalization
Top 5 assets by market cap (Billions USD).
Architecture & Data Flow
1
Extract & Transform(Python)
Fetches real-time data from API using Python requests library. Cleans, normalizes nested JSON, and adds timestamps using Pandas DataFrames.
2
Load (Neon Postgres)
Upserts processed data into Serverless PostgreSQL via SQLAlchemy engine.
3
Automate (GitHub Actions)
Orchestrates the pipeline to run daily via scheduled Cron jobs (Serverless CI/CD).
4
Visualize (Next.js and Recharts)
Displays live and historical data using a Next.js frontend and Recharts with dynamic Server Components.
Live Market Data (DB)
● Live Connection| Rank | Asset | Price (USD) | Market Cap | Updated (UTC) |
|---|---|---|---|---|
| #1 | BitcoinBTC | $88,248.00 | $1,763B | 1/27/2026, 12:57:45 AM |
| #2 | EthereumETH | $2,916.56 | $352B | 1/27/2026, 12:57:45 AM |
| #3 | TetherUSDT | $0.9989 | $186B | 1/27/2026, 12:57:45 AM |
| #4 | BNBBNB | $876.32 | $119B | 1/27/2026, 12:57:45 AM |
| #5 | XRPXRP | $1.90 | $116B | 1/27/2026, 12:57:45 AM |
| #6 | USDCUSDC | $0.9996 | $72B | 1/27/2026, 12:57:45 AM |
| #7 | SolanaSOL | $123.95 | $70B | 1/27/2026, 12:57:45 AM |
| #8 | TRONTRX | $0.2959 | $28B | 1/27/2026, 12:57:45 AM |
| #9 | Lido Staked EtherSTETH | $2,917.08 | $27B | 1/27/2026, 12:57:45 AM |
| #10 | DogecoinDOGE | $0.1221 | $21B | 1/27/2026, 12:57:45 AM |
| #11 | Figure HelocFIGR_HELOC | $1.03 | $15B | 1/27/2026, 12:57:45 AM |
| #12 | CardanoADA | $0.3516 | $13B | 1/27/2026, 12:57:45 AM |
| #13 | Wrapped stETHWSTETH | $3,575.36 | $12B | 1/27/2026, 12:57:45 AM |
| #14 | Bitcoin CashBCH | $577.98 | $12B | 1/27/2026, 12:57:45 AM |
| #15 | WhiteBIT CoinWBT | $53.82 | $12B | 1/27/2026, 12:57:45 AM |
| #16 | Wrapped BitcoinWBTC | $88,132.00 | $11B | 1/27/2026, 12:57:45 AM |
| #17 | Wrapped Beacon ETHWBETH | $3,177.22 | $11B | 1/27/2026, 12:57:45 AM |
| #18 | ChainlinkLINK | $13.72 | $10B | 1/18/2026, 12:58:40 AM |
| #19 | Wrapped eETHWEETH | $3,171.13 | $10B | 1/27/2026, 12:57:45 AM |
| #20 | MoneroXMR | $502.68 | $9B | 1/25/2026, 1:00:11 AM |
| #21 | USDSUSDS | $0.9996 | $9B | 1/27/2026, 12:57:45 AM |
| #22 | Binance Bridged USDT (BNB Smart Chain)BSC-USD | $0.9991 | $9B | 1/27/2026, 12:57:45 AM |