Intelligent News ETL Pipeline

An automated multi-agent system utilizing LLMs to transform unstructured global news into structured, platform-ready insights.

Live Insights: Past 7 Days
● Live Sync
Thu, Mar. 26, 2026 - Global Economic Shocks & WTO Investment Progress
Mar 26, 2026
(1/3) Global economy alert! 🚨 Middle East conflicts are disrupting supply chains & remittances, while a new WTO Investment Facilitation for Development (IFD) agreement seeks to streamline global investment. Key updates below 👇 (2/3) The ongoing Middle East conflict 🌏 is causing significant global supply chain disruptions 🚢 and creating economic risks for remittance flows. International bodies are urging action to lessen these widespread impacts. (3/3) Meanwhile, the Investment Facilitation for Development (IFD) Agreement has been finalized 🤝, setting the first global rules to streamline investment. Members are now pushing for its inclusion in the WTO framework 🏛️. #GlobalPolitics #TradePolicy #WTO #EconomicImpact #Investment
Tue, Mar. 24, 2026 - Iran Conflict's Global Economic Threat
Mar 24, 2026
(1/2) The escalating Iran conflict is a "major threat" to the global economy, warns the IEA chief, drawing parallels to past energy crises. Brace for potential oil price spikes & LNG supply tightness. 👇 (2/2) This geopolitical tension 🌍 risks further instability in energy markets 💹, impacting global economic recovery. Comparisons are made to the 1970s energy crisis & the 2022 Russia-Ukraine conflict. #IranConflict #GlobalEconomy #EnergySecurity #OilPrices #LNGSupply
Mon, Mar. 23, 2026 - Global Energy Crisis & Economic Threat
Mar 23, 2026
(1/3) Global economy faces a "significant threat" from the escalating Middle East energy crisis, with the Strait of Hormuz tensions raising alarm bells worldwide. India also acknowledges the severe impact. 👇 (2/3) The IEA warns that Middle East conflict, especially around the Strait of Hormuz, poses a "significant threat" to global economic stability. Its ripple effects will be felt by all nations. 🌏⛽️ (3/3) India's Cabinet Committee on Security (CCS) has recognized the "significant impact" of the West Asia conflict on both the global and Indian economies. Acknowledging the widespread ramifications. 🇮🇳💰 #GlobalEconomy #EnergyCrisis #MiddleEast #Geopolitics #India
Sun, Mar. 22, 2026 - Global Economic Slowdown & Geopolitical Risks
Mar 22, 2026
(1/4) Global economy faces a synchronized slowdown amid geopolitical conflicts, challenging central banks and risking critical supply chains. Get the breakdown 👇 (2/4) Global economy braces for a synchronous slowdown 📉🌏. Conflicts are impacting manufacturing & services worldwide, signaling weaker performance ahead. (3/4) Central banks grapple with policy challenges amidst growth risks 🏛️💹. The Fed hints at one rate cut this year, while the Bank of Canada holds steady. Monetary policy navigating choppy waters. (4/4) Escalating Middle East conflict threatens global energy & fertilizer supplies ⛽🌾. This could trigger a wider crisis, adding immense pressure to already strained markets. #GlobalEconomy #Geopolitics #CentralBanks #SupplyChain #EconomicOutlook
Sat Mar 21 2026 - Middle East Tensions & Global Impact
Mar 21, 2026
(1/4) Global focus on the Middle East as escalating tensions in the Strait of Hormuz ripple through shipping and energy markets, impacting supply chains worldwide. Norway weighs in on international law. 👇 (2/4) The Middle East conflict intensifies, driving up shipping risk premiums through the vital Strait of Hormuz 🌊. This poses significant challenges for global maritime trade and commodity flows. 🚢 (3/4) Supply chains are feeling the pinch, with examples like India's fertilizer imports facing disruptions. 🇮🇳 Asia's major energy importers are now urgently reassessing their long-term energy security strategies amidst the volatility. ⛽ (4/4) Adding to the geopolitical landscape, Norway has stated that attacks against Iran do not comply with international law. 🏛️ This highlights the complex legal and diplomatic dimensions of the ongoing regional conflict. #MiddleEastConflict #GlobalShipping #EnergySecurity #Geopolitics #InternationalLaw
Fri Mar 20 2026 - Geopolitical Tensions & Trade Dynamics
Mar 20, 2026
(1/4) Global geopolitical tensions are soaring today as the Iran crisis escalates, challenging international alliances and stirring trade disputes. Here's a quick rundown 👇 (2/4) 1. The #Iran crisis intensifies 🇮🇷, fueling global oil market volatility ⛽. This escalation is putting significant pressure on 21st-century alliances, prompting a re-evaluation of international partnerships and their limits. (3/4) 2. Amidst rising regional tensions, Japanese PM Takaichi Sanae met with US President Trump 🇯🇵🇺🇸. This high-level dialogue underscores the critical need for key allies to coordinate responses to global challenges. #Diplomacy (4/4) 3. In India, opposition is mounting against the proposed India-US trade deal 🇮🇳. Farmer groups & civil society are planning major protests, highlighting domestic concerns over trade agreements. #TradeWars #IndiaUS #Protests #GlobalEconomy

Pipeline Architecture

1
Incremental Extraction

Uses API to fetch global data. Implements State Tracking in cloud DB to ensure only new articles since the last successful run are processed.

2
AI Transformation (Analyst)

Leverages Gemini 2.5 Flash to perform semantic filtering, removing noise and summarizing high-impact politico-economic events.

3
Structured Loading (Writer)

A dedicated agent converts draft summaries into Schema-validated JSON, optimized for multi-platform distribution (e.g., X/Twitter threads).

4
Persistence & Orchestration

Stores processed content in PostgreSQL DB. Entire workflow is orchestrated via GitHub Actions on a serverless daily schedule.

LLM-as-a-Transformer

Solves the challenge of cleaning highly heterogeneous news data where traditional Regex or rule-based engines fail.

Robust State Management

Prevents data duplication and optimizes API costs by persisting lastRunTimestamp across serverless execution environments.

Ethical Data Design

Engineered to include original source attribution and "Transformative Use" logic to adhere to content copyright standards.