🔦Data Aggregation

Challenges of Data Aggregation

For those who wish to use data with any level of seriousness, handling data sources is often more challenging than expected.

1. Access Authorization: One must have the necessary permissions to access the data.

2. Proficiency in Interfaces: One needs to be skilled in using the interfaces to extract the desired data. If these interfaces are not user(developer)-friendly, the difficulty level increases exponentially.

3. Trustworthiness of Data Sources: There must be a reasonable assumption that the data source is reliable.

4. Managing Multiple Data Sources: Diversifying data sources to avoid dependence on a single source introduces management complexities.

Now, imagine if this data needs to be consensus-based. The complexity can be beyond imagination.

Different Data Handling Methods and Content Across Data Sources

Data sources are created by various entities, organizations, or individuals. They collect, publish, sell, and share data using their own logic. Typically, this logic operates on cloud-based servers like AWS.

This is where problems arise:

1. Different Data Manipulation Methods: Each data provider sets their own method for data manipulation.

2. Variability in Data: Even for the same type of data, the actual values can vary when queried.

For example, have you ever checked exchange rates? Each data provider has slight discrepancies.

We constantly need to consider which data to trust and how to handle the vast amount of data available.

Architecture Integrates These Data Sources

What if we could integrate data sources to solve all the aforementioned problems and even make them convenient to use? This would be revolutionary.

Advantages include:

• You only need to learn a single, simple ‘unified data manipulation method.’

• The system automatically determines trustworthy data sources (or allows for customization by the requester).

This is extremely easy! Because all you need to do is call Dagg.read("desired data").

With pre-configured unified request timing for source data and consistent data synthesis methods for different types of data, the previously mentioned problems naturally dissolve.

What Data Sources Do We Provide?

We provide data from prominent players in the DeSoc realm, including Neynar, Airstack, and other major hubs. For more detailed information, please check the following sections.

🔦DAGG - Data Source Aggregator