AI-Powered Market Research Automation System

The problem

The client, owner of a market research firm, sought to enhance research capabilities using AI-driven automation. The project began with a discovery call to understand requirements, particularly around tracking market signals in the Mexican beer industry. A system was designed to automate data collection from multiple sources including news platforms and online communities. Using tools like Apify for scraping and Make for automation, the workflow captures raw data, processes it through AI for summarization and insights, and stores structured outputs in Airtable. While a client-facing dashboard via Softr was initially considered, the strategy shifted toward delivering curated insights directly via email for a more personalized approach. The system was built with scalability in mind, allowing future integration of dashboards and expanded data sources.

What we built

This project involved building a comprehensive AI-powered market research automation system tailored for a consulting firm. The objective was to streamline the process of gathering and analyzing market signals, particularly in niche industries such as the Mexican beer market.

The solution leverages Apify to scrape data from sources like Google News, Reddit, and potentially social media platforms such as X, Instagram, Facebook, and LinkedIn. These data points are then passed through Make (Integromat) workflows, where automation pipelines process and organize the information. AI is used to generate summaries, extract key entities such as brands and competitors, and structure insights for further analysis.

All processed data is stored in Airtable, which acts as the central database. It includes structured fields such as signal name, raw content, summaries, competitor mentions, and consultant insights. The system also supports targeted scraping (e.g., specific subreddits) to ensure relevance and avoid noise from ambiguous keywords.

While an initial plan included building a client-facing dashboard using Softr, the approach was adjusted to prioritize personalized delivery of insights via email. This allows the consultant to add a strategic point of view before sharing findings with clients. The system is designed to be modular and scalable, allowing future expansion into dashboards and additional integrations.