Automated Crypto Data Pipeline
A full-stack data engineering project demonstrating an end-to-end automated ETL pipeline.
Market Analytics
Top 5 Assets Performance (%)
Relative performance rebased to 0% at the start of the period. Comparing XRP, BTC, BNB, ETH, USDT.
Market Capitalization
Top 5 assets by market cap (Billions USD).
Architecture & Data Flow
1
Extract & Transform (Python)
Fetches real-time data from API using Python requests library. Cleans, normalizes nested JSON, and adds timestamps using Pandas DataFrames.
2
Load (Neon Postgres)
Upserts processed data into Serverless PostgreSQL via SQLAlchemy engine.
3
Automate (GitHub Actions)
Orchestrates the pipeline to run daily via scheduled Cron jobs (Serverless CI/CD).
4
Visualize (Next.js and Recharts)
Displays live and historical data using a Next.js frontend and Recharts with dynamic Server Components.
Live Market Data (DB)
Live Connection| Rank | Asset | Price (USD) | Market Cap | Updated (Local) |
|---|---|---|---|---|
| #1 | BitcoinBTC | $81,055.00 | $1,628B | 9/20/2026, 2:04:30 AM |
| #2 | EthereumETH | $2,618.68 | $320B | 9/20/2026, 2:04:30 AM |
| #3 | TetherUSDT | $0.9996 | $183B | 9/20/2026, 2:04:30 AM |
| #4 | BNBBNB | $760.80 | $101B | 9/20/2026, 2:04:30 AM |
| #5 | XRPXRP | $1.40 | $88B | 9/20/2026, 2:04:30 AM |
| #6 | USDCUSDC | $0.9997 | $74B | 9/20/2026, 2:04:30 AM |
| #7 | SolanaSOL | $110.14 | $65B | 9/20/2026, 2:04:30 AM |
| #8 | TRONTRX | $0.3401 | $32B | 9/20/2026, 2:04:30 AM |
| #9 | ZcashZEC | $1,471.87 | $25B | 9/20/2026, 2:04:30 AM |
| #10 | Figure HelocFIGR_HELOC | $1.02 | $23B | 9/20/2026, 2:04:30 AM |
| #11 | Lido Staked EtherSTETH | $2,240.59 | $21B | 2/4/2026, 1:02:01 AM |
| #12 | HyperliquidHYPE | $91.67 | $20B | 9/20/2026, 2:04:30 AM |
| #13 | DogecoinDOGE | $0.08719 | $14B | 9/20/2026, 2:04:30 AM |
| #14 | MoneroXMR | $538.08 | $10B | 9/20/2026, 2:04:30 AM |
| #15 | Wrapped stETHWSTETH | $2,742.38 | $10B | 2/4/2026, 1:02:01 AM |
| #16 | WhiteBIT CoinWBT | $82.64 | $10B | 9/20/2026, 2:04:30 AM |
| #17 | RainRAIN | $0.01372 | $10B | 9/20/2026, 2:04:30 AM |
| #18 | USDSUSDS | $0.9998 | $10B | 9/20/2026, 2:04:30 AM |
| #19 | Wrapped BitcoinWBTC | $75,650.00 | $9B | 2/4/2026, 1:02:01 AM |
| #20 | ChainlinkLINK | $12.31 | $9B | 9/20/2026, 2:04:30 AM |
| #21 | Binance Bridged USDT (BNB Smart Chain)BSC-USD | $0.9982 | $9B | 2/4/2026, 1:02:01 AM |
| #22 | Wrapped eETHWEETH | $2,938.44 | $9B | 1/31/2026, 1:00:12 AM |
| #23 | CardanoADA | $0.2273 | $9B | 9/20/2026, 2:04:30 AM |
| #24 | LEO TokenLEO | $8.91 | $8B | 9/20/2026, 2:04:30 AM |
| #25 | Wrapped Beacon ETHWBETH | $2,437.66 | $8B | 2/4/2026, 1:02:01 AM |
| #26 | StellarXLM | $0.1946 | $7B | 9/20/2026, 2:04:30 AM |
| #27 | Bitcoin CashBCH | $333.94 | $7B | 5/28/2026, 1:51:44 AM |
| #28 | RaveDAORAVE | $26.11 | $6B | 4/18/2026, 1:15:22 AM |
| #29 | ToncoinTON | $2.31 | $6B | 5/13/2026, 1:56:54 AM |
| #30 | MemeCoreM | $4.74 | $6B | 4/24/2026, 1:24:17 AM |
| #31 | LABLAB | $19.07 | $6B | 6/27/2026, 2:00:34 AM |
| #32 | Ethena USDeUSDE | $0.9994 | $6B | 4/19/2026, 1:24:28 AM |
| #33 | CantonCC | $0.1426 | $6B | 7/1/2026, 2:11:21 AM |