Intelligent News ETL Pipeline

An automated multi-agent system utilizing LLMs to transform unstructured global news into structured, platform-ready insights.

Live Insights: Past 7 Days
● Live Sync
Mon, Jan. 26, 2026 - Global Geopolitics & Economic Shifts
Jan 26, 2026
(1/4) 🚨 Global Briefing: US tightens foreign investment rules, G-7 rallies for Ukraine's energy, and Saudi Arabia rethinks its ambitious Neom project. Major shifts in geopolitics & economics are unfolding! 👇 (2/4) 🇺🇸 US Congress is scrutinizing a $5.5B acquisition of a tech firm with sensitive military contracts. This signals new 'red lines' for foreign investment in critical sectors, emphasizing national security. 🛡️💹 (3/4) 🇫🇷 France will host G-7 finance ministers to coordinate crucial international support for Ukraine's energy grid, battered by Russian attacks. A united front to bolster resilience. 🇺🇦⚡ (4/4) 🇸🇦 Saudi Arabia's ambitious Neom 'The Line' project is set for a significant scale-back & redesign due to financial pressures. Reality check for mega-projects. 🏗️📉 #Geopolitics #GlobalEconomy #USInvestment #UkraineAid #SaudiVision
Sun Jan 25 2026 - Global Economic & Political Shifts
Jan 25, 2026
(1/4) Major global shifts this week: Japan's bond yields hit records, Libya secures a huge oil deal, and Trump issues a stark trade warning to Canada. Dive in for the details 👇 (2/4) Japan's government bond yields soared to historic highs this week 🇯🇵💹, sending ripples across global financial markets. Investors are closely watching the implications for monetary policy and stability. (3/4) Libya has inked a significant 25-year oil development deal for the Waha field with TotalEnergies & ConocoPhillips 🇱🇾⛽. This aims to substantially boost the nation's oil production 📈, impacting global energy supplies. (4/4) Former US President Trump has threatened a massive 100% tariff on Canadian imports 🇺🇸🇨🇦 if Canada pursues a trade agreement with China 🇨🇳🚫. This signals potential new trade tensions on the horizon. #GlobalPolitics #TradeWars #JapanEconomy #LibyaOil #USCanadaRelations
Sat, Jan. 24, 2026 - Global Tensions & Trade Shifts
Jan 24, 2026
(1/3) Global markets are buzzing! Geopolitical tensions are driving oil prices up, while major trade deals are on the horizon. But what does it all mean for consumers? Dive into today's top economic shifts. 👇 (2/3) 📈 Geopolitical tensions in the Middle East & Eastern Europe have sent bullish bets on Brent crude to a 9-month high. Energy markets are closely watching for further developments. 🌍⛽ (3/3) Meanwhile, the EU & India are nearing a landmark trade agreement 🤝🇮🇳, signaling potential new economic avenues. Trade wars show signs of cooling, but analysts warn consumer impacts will emerge this year. 🛍️📉 #GlobalEconomy #TradeDeals #OilMarkets #Geopolitics #ConsumerImpact
Fri, Jan. 23, 2026 - Brazil's Will Bank Liquidation & US-NATO Greenland Deal
Jan 23, 2026
(1/3) Major shifts in global finance & geopolitics today! Brazil's fintech sector faces a jolt with Will Bank's liquidation, while the US & NATO reach a 'framework agreement' on Greenland. Dive into the details 👇 (2/3) Brazil's Central Bank has ordered the liquidation of fintech firm Will Bank. 🇧🇷💸 This move comes despite a previous acquisition attempt by Abu Dhabi's sovereign wealth fund, Mubadala. A significant development for LatAm's financial tech landscape. (3/3) At WEF Davos, President Trump announced a 'framework agreement' with NATO on Greenland. 🇺🇸🇩🇰🏛️ This follows Denmark's rejection of a US acquisition bid & Trump retracting earlier military threats. Diplomacy prevails after a period of tension. #GlobalPolitics #FintechNews #USDiplomacy #BrazilEconomy
Thu Jan 22 2026 - Global Geopolitics & Economic Outlook
Jan 22, 2026
(1/4) Today's top global stories: Trump's Arctic diplomacy, tokenized money's rise, and a looming oil surplus. Dive in! 👇 (2/4) 🇺🇸 Greenland 🇬🇱: President Trump secured a framework agreement on Greenland's future, aiming to de-escalate tensions in the strategic Arctic region. A key diplomatic move. 🧊 (3/4) 🇮🇹 Finance 💹: Italy's Central Bank Governor suggests commercial bank money could be tokenized in the future, signaling a potential shift in digital finance. The future of currency? 💰 (4/4) ⛽ Energy 🌍: The IEA predicts a significant global oil market surplus for Q1 2026, pointing to potential price impacts & market shifts. Keep an eye on crude! 📉 #GlobalPolitics #Economics #EnergyMarkets #ArcticDiplomacy #DigitalFinance
Wed, Jan. 21, 2026 - Global Economic & Political Updates
Jan 21, 2026
(1/3) Major global shifts today: Goldman Sachs expands in Qatar, EU-US trade hits a snag over tariff threats, and the EU-Mercosur deal faces EPP opposition. Dive into the latest international economic & political developments! 👇 (2/3) Goldman Sachs is deepening its ties with Qatar's sovereign wealth fund in a massive $25 billion deal. 🤝 The firm is also developing its Doha office into a key regional hub. 💰 Expect increased investment activity from the Gulf! 🇶🇦 (3/3) Transatlantic trade tensions rise as the EU halts approval of its trade deal with the US, citing President Trump's tariff threats. 🇪🇺🇺🇸 Meanwhile, the European People's Party (EPP) opposes sending the EU-Mercosur agreement to the European Court of Justice. 🏛️ #GlobalPolitics #TradeWars #EUUSRelations #QatarInvestment #Mercosur
Tue, Jan. 20, 2026 - US-EU Trade Tensions Escalate, Global Trade War Risk
Jan 20, 2026
(1/3) Global trade tensions are soaring as US-EU relations sour over new tariff threats, sparking fears of a full-blown trade war and economic instability worldwide. 👇 (2/3) President Trump's recent tariff warnings against the EU are met with strong backlash. The European Union is now considering retaliatory tariffs, escalating the dispute. 🇺🇸🇪🇺🛡️ (3/3) Experts warn that this transatlantic trade spat could trigger a global trade war, leading to significant economic turmoil and instability across markets. Stay informed. 💹🌏💥 #USTrade #EUTariffs #GlobalEconomy #TradeWar #Geopolitics

Pipeline Architecture

1
Incremental Extraction

Uses API to fetch global data. Implements State Tracking in cloud DB to ensure only new articles since the last successful run are processed.

2
AI Transformation (Analyst)

Leverages Gemini 2.5 Flash to perform semantic filtering, removing noise and summarizing high-impact politico-economic events.

3
Structured Loading (Writer)

A dedicated agent converts draft summaries into Schema-validated JSON, optimized for multi-platform distribution (e.g., X/Twitter threads).

4
Persistence & Orchestration

Stores processed content in PostgreSQL DB. Entire workflow is orchestrated via GitHub Actions on a serverless daily schedule.

LLM-as-a-Transformer

Solves the challenge of cleaning highly heterogeneous news data where traditional Regex or rule-based engines fail.

Robust State Management

Prevents data duplication and optimizes API costs by persisting lastRunTimestamp across serverless execution environments.

Ethical Data Design

Engineered to include original source attribution and "Transformative Use" logic to adhere to content copyright standards.