Big Data Technology Components :
1. Ingestion :
The ingestion layer is the very first step of pulling in raw data.
It comes from internal sources, relational databases, non-relational databases, social media, emails, phone calls etc.
There are two kinds of ingestions :
Batch, in which large groups of data are gathered and delivered together.
Streaming, which is a continuous flow of data. This is necessary for real-time data analytics.
2. Storage :
Storage is where the converted data is stored in a data lake or warehouse and eventually processed.
The data lake/warehouse is the most essential component of a big data ecosystem.
It needs to contain only thorough, relevant data to make insights as valuable as possible.
It must be efficient with as little redundancy as possible to allow for quicker processing.
3. Analysis :
In the analysis layer, data gets passed through several tools, shaping it into actionable insights.
There are four types of analytics on big data :
- Diagnostic: Explains why a problem is happening.
- Descriptive: Describes the current state of a business through historical data.
- Predictive: Projects future results based on historical data.
- Prescriptive: Takes predictive analytics a step further by projecting best future efforts.
4. Consumption :
The final big data component is presenting the information in a format digestible to the end-user.
This can be in the forms of tables, advanced visualizations and even single numbers if requested.
The most important thing in this layer is making sure the intent and meaning of the output is understandable.
0 Comments