Project FAQs
About us, Products, Technology, Roadmap FAQs
About Openmesh
Q: What is Openmesh?
A: Openmesh is an initiative to build an all-in-one petabyte scale decentralized open data infrastructure for web3 allowing anyone access to immutable data without any censorship.
Q: What can Openmesh do for the community?
A: Openmesh will enable anyone to stream live web3 transactional data, access historical web3 data, query and perform analytics, build interoperable data products, and do business intelligence, at scale, at almost zero cost forever without having any intervention, interference, or influence over regulations, governments, corporations including the creators themselves.
Q: What is Openmesh's vision?
A: Openmesh will serve as the “Wayback Machine” and “Open Finance Archive of Web 3.0”. We envision a thriving ecosystem of researchers, data scientists, entrepreneurs, quants, traders, financial engineers, protocol architects, startups, lawyers, and regulators using Openmesh Open Data Infrastructure for live and unedited, tamper-proof historical information to build various applications from research & data analytics, trading, risk management, financial engineering, financial forensics, etc.
Q: What is Openmesh's mission?
A: Openmesh's mission is to address the information asymmetry in Web 3.0 by democratizing data and providing everyone with access to quality, immutable data for free, forever.
Q: How did Openmesh start?
A: Openmesh has been in the blockchain and crypto space since 2015 and faced issues with data quality and cost from centralized data providers. The team decided to build its own data lake, which took more than a year to build, but now covers 70% of the entire crypto and Web 3.0 transactional data market.
Q: What types of data does Openmesh store and cover? What are the sources of these data?
A: Openmesh stores all transactional data in Web 3.0 including decentralized finance data, decentralized crypto exchange data, centralized crypto exchange data, blockchain game data, metaverse data, oracle data, and public blockchain data.
| Source | Type of data |
|---|---|
Decentralized Finance | All financial transactions including wallets, assets, liquidity, value locked, asset transfers, asset custody |
Decentralized Crypto Exchanges | All financial transaction, liquidity, assets, asset transfers, wallets |
Centralized Crypto Exchanges | Order books, trade data, transactional data, candle stick, ticker, inflow and outflow data |
Blockchain Games | Games, user assets, user activities, game asset classifications, game assets transfers, in-game transactions |
Metaverses | Metaverse assets, transactions, user activities
|
Oracles | All oracle transactions |
Public Blockchains | Block size, block events, asset transfers, wallet addresses, blockchain confirmations, smart contract events |
More information about Web 3.0 data can be found here on our medium page.
Q: What is the current status of Openmesh?
A: Openmesh’s current infrastructure v4 covers approximately 80% of the entire crypto and Web 3.0 transactions and is collecting 0.8-2.5 terabytes of data per day. Openmesh estimates that it will soon reach the petabyte scale.
Q: What is Openmesh's approach to transparency?
A: Openmesh is transparent in all aspects, including open-source technology, open accounting, open R&D, open infrastructure monitoring, and open verification through end-to-end data encryption.
Q: What is Openmesh's approach to excellence and passionate maximalism?
A: Openmesh's team works 7 days a week with extreme passion and discipline toward innovation in the Web 3.0 and open-source ecosystem.
Q: What is Openmesh's philosophy?
A: Openmesh's philosophy is based on decentralization, with a belief in openness, equality, and freedom.
Q: How do we aim to make Openmesh work?
A: Openmesh aims to use a self-serve design and data mesh architecture that supports distributed, domain-specific data consumers and views data as a product.
Q: How does Openmesh handle data?
A: Openmesh has no single authority that can alter its data, how the information is distributed, or who can access it. The infrastructure is open-source and governed by an open-source community, and each domain handles its own data pipelines.
Q: How is the data aimed to be stored and managed?
A: The data is stored in a decentralized data lakehouse secured by cryptography
Q: Who are our users?
Individuals (Quants, developers, metaverse engineers)
Enterprise (Governments, Regulators, Law firms, VC Firms)
Products
Openmesh Streaming services
Q: What is a Live Web 3.0 data streaming service?
A: A service that allows users to subscribe to live data feeds, such as Coinbase BTC/USD.
Q: Can end users access the service for free?
A: Yes, end users can subscribe to live data feeds for free.
Q: What type of data feed is available for subscription?
A: An example of a data feed available for subscription is the Coinbase BTC/USD feed.
Q: How to access the streaming service?
Through our WebSocket API endpoint
Openmesh Historical data-as-a-service
Q: What is Historical data-as-a-service?
A: Anyone can search historical data through Openmesh PowerQuery.
Q: How can end users request historical transactions?
A: End users can request historical transactions through our Unified API/ Rest API or our Powerquery, and GraphQL.
Q: What type of data is available for request?
A: All web3 transactional data in the infrastructure is available for free. More information about our feeds could be found here.
An example of data available to request is all BTC/USD trade data on Coinbase between 10-10-2019 to 01-02-2020.
Openmesh PowerQuery
Q: What is an Analytical & query service?
A: A service that allows users to query, combine and run analytics on data via a web-based open analytical application.
Q: What can users do with the service?
A: Users can create, test, visualize, and deploy custom data feeds, develop data products, and gain insights at scale.
Q: Can users perform real-time analysis with the service?
A: Yes, an example of real-time analysis is the net inflow of USDT on Uniswap compared to Binance in the last 26 minutes.
Q: What is the goal of the service?
A: The goal is to introduce on-chain analytics by plugging the query engine directly into smart contracts, enabling a data-driven web3 industry.
Openmesh GraphQL
Q: What can Openmesh GraphQL do?
A: We created this GraphQL PoC to demonstrate to our community what our idea is for a server that interacts with our data lake. It is not meant for scale or is the ideal end result for a production application, rather it is only our starting point.
Q: What is GraphQL?
A: GraphQL is a query language for APIs and a runtime for executing those queries.
Q: What is the benefit of using GraphQL on Openmesh?
A: Using GraphQL on Openmesh allows for flexible and efficient querying of data, with the ability to specify the exact data needed, reducing the amount of data transferred over the network.
Q: What is the data source for GraphQL on Openmesh?
A: The data source for GraphQL on Openmesh is Openmesh's centralized and decentralized data infrastructure.
Q: What are some of the features of GraphQL on Openmesh?
A: Some features of GraphQL on Openmesh include auto-generated GraphQL schemas, real-time data streaming, data filtering and sorting, and support for pagination.
Q: How does Openmesh ensure the privacy of data accessed through GraphQL?
A: Openmesh ensures the privacy of data accessed through GraphQL by implementing end-to-end encryption for data collection, processing, and streaming.
Q: What are some potential use cases for GraphQL on Openmesh?
A: Potential use cases for GraphQL on Openmesh include real-time market data analytics, algorithmic trading, and developing decentralized applications (dApps) that require access to real-time market data.
Q: What is the benefit of using Openmesh's data infrastructure with GraphQL?
A: Using Openmesh's data infrastructure with GraphQL provides a robust and reliable data source for developers to build decentralized applications on top of, with support for real-time data streaming and querying.
Q: How can developers get started with using GraphQL on Openmesh?
A: Developers can get started with using GraphQL on Openmesh by exploring the Openmesh GraphQL API documentation, which provides detailed information on how to query data using GraphQL.
Q: Where can I see the repo?
A: https://github.com/L3A-Protocol/graphql-poc
Technology
Unified API
Q: What is Openmesh Unified API?
A: Openmesh provides a unified and standardized interface for accessing data from different sources.
Q: How was Openmesh Unified API built?
A: The WebSocket service for L3 Atom is designed for use cases that require low latency and real-time data. As the market and on-chain events are processed through our data pipeline, they are emitted over the Websockets protocol via a single endpoint. Horizontally scaled broadcasters subscribe to all Kafka topics which contain normalized data and stream incoming data to users via AWS API Gateway, providing a scalable and fault tolerant system to accommodate intense requirements.
More information could be found here.
Universal Data Collector
Q: What is the Universal Data Collector?
A: The Universal Data Collector (UDC) is a core component of the Openmesh. It is designed to facilitate the collection, enhancement, and standardization of a variety of data sources, extracting atomic events and packaging them into indexed data sets that can be used for live-streamed and historical data analytics.
Q: How does the UDC collect data?
A: The UDC collects raw data by connecting to centralized exchange APIs (both HTTP and Websockets) and blockchain nodes. It pulls market data directly from the source, and blockchain data collectors make JSON RPC calls to pull full, granular data to be processed and transformed into human-readable on-chain events.
Q: What happens to the data after it's collected?
A: After the raw data is collected, it's produced to an Apache Kafka cluster, where a network of stream processors pull the data and process the raw data into standardized events. The raw data is filtered into different categories – first by exchange, then by event type. After the events are filtered, their schemas are transformed to a standardized format, which are then produced back to Kafka to their appropriate topics.
Q: How is the UDC monitored?
A: The UDC is monitored via Prometheus, with a public Grafana dashboard that the community can use to gain telemetry into the infrastructure.
Q: Can community members expand L3 Atom's data coverage?
A: Yes, the UDC has been designed to make it as easy as possible for community members to expand L3 Atom's data coverage. The high-level steps to add an additional data source are: define the unique connectivity process, define the normalization procedures, and submit a pull request. If accepted, the data source will be introduced to L3 Atom as another node on the network.
Q: What are the benefits of using the UDC?
A: The UDC facilitates the collection, enhancement, and standardization of a variety of data sources, allowing for live-streamed and historical data analytics. It is fault-tolerant and handles unexpected events from the exchange that could cause an outage or drop in data. It is designed to work with any connection to blockchain nodes and doesn't rely on existing APIs. Finally, being an open-source project, it is highly modularized and containerized, making it easy to expand L3 Atom's data coverage.
More information could be found here.
Openmesh v4 Roadmap
Q: What is Openmesh v4?
A: Openmesh v4 is a significant milestone for the team, filled with R&D verticals, aimed at improving the data infrastructure.
Q: What are some of the R&D plans in the pipeline for Openmesh v4?
A: Some of the R&D plans in the pipeline for Openmesh v4 include migrating to a decentralized network of computing and storage, end-to-end encryption, PowerQuery for querying large datasets, expanding data coverage, introducing governance and protocol defensibility framework, high availability, modern data architecture, on-chain queryability, indexing with open NLP, introducing data standards and OpenMetrics, and Fusion integrated development environment.
Q: What is the focus on data privacy in Openmesh v4?
A: The focus on data privacy in Openmesh v4 is end-to-end encryption for data collection, processing, and streaming, including queries.
Q: How will Openmesh v4 expand its coverage?
A: Openmesh v4 plans to expand its coverage to 50 centralized exchanges, 15 decentralized exchanges, 10 DEX aggregators, 100 protocols, and 20 blockchains.
Q: What is the governance mechanism in Openmesh v4?
A: Openmesh v4 introduces a governance framework for the core infrastructure and Universal Data Collectors with a consensus mechanism designed to align the present state of the data with the state of the core infrastructure. No single authority, including the founding team, can alter the data or the infrastructure.
Q: What is the focus on service availability in Openmesh v4?
A: The focus on service availability in Openmesh v4 is high availability by designing redundancy systems and expanding the v3 architecture to different regions.
Q: What is the modern data architecture in Openmesh v4?
A: The modern data architecture in Openmesh v4 is a data mesh architecture, where data is treated as a product and anyone can design, deploy, and provision data applications without worrying about cloud infrastructure, servers, locations, licensing, etc.
Q: What is on-chain queryability in Openmesh v4?
A: On-chain query ability in Openmesh v4 allows connecting data feeds, indexes, metadata, and data products directly to smart contracts, enabling Web 3.0 developers and protocols to make data-driven decisions, develop smart contract automation, and run on-chain analytics.
Q: What is Fusion in Openmesh v4?
A: Fusion in Openmesh v4 is an integrated development environment that enables programmers, quants, and financial engineers to consolidate, create, build, test, deploy, integrate, and manage decentralized financial applications all in one place.
Last updated