Background
There is no doubt that many businesses now use sophisticated systems for decision-making but their great limitation is in finding and accessing good quality data. The Fintech AI Innovation Consortium is a non-profit organisation that will use students to build an Event Intelligence Application using agile methods combined with cloud and DevOps principles so that different components can be built progressively over time. Teams will be following agile processes to build the system incrementally and comply with some design recommendations.
The Event Intelligence Application aims to help users create “event datasets“ and offer them to researchers and small companies so that they can conduct analytics experiments or feed into their business intelligence systems. As an example, an investment company wants to look at weather events because bad weather can influence the production of agricultural goods which in turn affects the stock price of companies that distribute these goods. Another example is looking at political events from the news that might affect some oil-producing countries which would increase transport costs.
Sometimes companies are interested in understanding chains of events: an event that causes another event to happen, which causes another event etc. This can be used to make predictions in the future. Examples of events relevant to users include:
Financial Events: Movements and trends in the financial markets, including changes in stock prices, currency exchange rates, and economic indicators that influence financial decisions and market behaviours.
Climate-Related Events: Changes in temperature, humidity, and other climate variables, potentially affecting regions or specific buildings.
News Events: Announcements or developments that impact society, the economy, or specific industries.
Social Media Events: Activities on social media platforms that signify significant events, trends, or public sentiment shifts.
Economic Events: Fluctuations in economic indicators such as interest rates and stock prices, on both macro (days/months) and micro (hours/minutes) scales.
Health-Related Events: Information about health crises, disease outbreaks, or advancements in medical research.
ESG (Environmental, Social, and Governance) Events: Data that reflects a company's operations to environmental stewardship, social responsibility, and governance practices. This includes corporate actions impacting sustainability, ethical impacts of business practices, social justice issues, and governance structures that ensure accountability and transparency. ESG data is critical for investors and stakeholders looking to assess the sustainability and ethical impact of their investments, as well as for companies aiming to improve their societal footprint.
In summary, the Event Intelligence Application is aimed at users interested in building datasets that help them do further processing (e.g., investigate some hypothesis, visualise events on a dashboard, make predictions etc.).
As many users can have overlapping needs, the company decides to gather as much event data as possible from different sources and store it in a data lake, then use different data processing pipelines to build custom datasets for different users. This way, the overall Event Intelligence Application automates the process of gathering raw data from several data sources and analysing the data using data processing pipelines made up of reusable components.
Sprints Overview
Sprint 1 will consist of creating an API microservice and testing it locally.
The list of external data sources is available here: 24T1-Additional information on available data sources
Each team can select an external data source and build an import API for this data source in Sprint 1.
Interested teams can also choose one/more publicly available datasets in the area of events and make them accessible as APIs to be used by other teams. The objective of this task is to utilize already pre-processed and acquired data in API design and development processes.
Sprint 2 will consist of another team conducting tests for the API developed in Sprint 1 within the shared infrastructure. A simple visualization client will be provided.
Sprint 3 will consist of teams building a visualisation application for viewing events.
IMPORTANT! Please carefully review the details in the project overview document: