introduction and system architecture added to the compiled docs

docs/COMPILED_ORACLE_DOCS.md

@@ -1,12 +1,79 @@
 # RedStone oracle technical doc
 
-## Introduction (problem statement)
+## Introduction
+
+### Overview
+Redstone is a data ecosystem that should deliver fast and accurate financial information in a decentralised fashion.
+
+#### Problem Statement (Defi Pain points)
+- It is very expensive to put all the pricing data on-chain (it cost more than 2m to do it for Ethereum Defi with Chainlink)
+- To reduce costs current providers cover only a small subset of tokens (Chainlink: 25) and have low update frequency (Chainlink: 10m)
+- DeFi protocols cannot expand beyond a small set of assets and cannot offer advanced solutions like margin lending (which require higher update frequency)
+
+#### Solution
+- Leverage Arweave blockchain as a cheap and permanent storage
+- Use token incentives to motivate data providers to maintain data integrity and the uninterrupted service
+- Use signed meta-transactions to deliver prices on-chain
+
+### Top level view
+The ecosystem could be divided into 3 main areas:
+
+- **Data provision** responsible for fetching the data from external api, transforming to a common format, and persisting collected information.
+  - Implemented as → [RedStone Node](https://github.com/redstone-finance/redstone-node)
+- **Data access** responsible for serving data to end user by various means including web portal, http api, discord bots, on-chain feeds or 3rd party applications (like spreadsheet extensions)
+  - Web portal → [RedStone App](https://github.com/redstone-finance/redstone-app)
+  - HTTP Api → [RedStone Api](https://github.com/redstone-finance/redstone-api)
+  - Bots → [Examples](https://github.com/redstone-finance/redstone-api/tree/main/examples/discord-bots)
+- **Data integrity** responsible for enforcing high quality of data by incentivising providers with tokens for keeping their service and punishing them for outage and misrepresented data
+  - Concept → [Argue protocol](https://github.com/redstone-finance/redstone-node/blob/main/docs/DISPUTE_RESOLUTION.md)
 
 
-## Architecture
+## System architecture
+
 ### Modules
+
+![redstone system architecture](img/redstone-system-architecture.png)
+
+#### External integrations (blue)
+- **Data sources** - provide data (like price information) via API
+- **DeFi protocols** - consume aggregated and attested data
+- **Web integrations** - utilise provided data to power apps like blockchain wallets, portfolio browsers, defi analytics sites
+
+#### Stakeholders (red)
+- **Data providers** - need to register with token stake and operate RedStone node which process and attest data
+- **End users** - may browse pricing data and select providers using the web portal
+- **Juries** - protect data integrity deciding if a price was manipulated
+
+#### RedStone modules (purple)
+- **RedStone node** - fetches data from external sources via api, aggregates and attest information with a cryptographic key, broadcast the results and persist them on the Arweave blockchain
+- **Data cache** - a module that holds data and makes it available to interested parties. Multiple services might implement it on various infrastructure to increase availability. It should achieve minimum latency and scale to handle large volume of requests
+- **Smart contracts api** - it provides the data to on-chain protocols. Initially, we cover Arweave and EVM (ethereum) infrastructure. It minimizes the running (gas) costs and verifies signed data on-chain
+- **HTTP api** - web based that serves information via REST api. It can power websites, node-js scripts and social media bots. There will be a dedicated java script wrapper that will abstract request formatting and error handling to make service easier to integrate
+- **Providers registry** - an Arweave smart contract that manages provider's token stake, holds the manifesto (SLA of pricing data) and allows checking historical performance (showing any data inconsistency and downtime)
+- **Dispute resolution protocol** - a set of Arweave contracts allowing to challenge any existing pricing feed, and managing the dispute process enabling juries to vote for the verdicts
+- **Web portal** - a web application that is an interface to browse data, check providers' statistics, see historical feeds with an option to raise a dispute and participate in the voting process
+
 ### Token design
 
+The token facilitates providing reliable and accurate information to blockchain networks from the external world.
+
+#### Usage of the token
+Tokens are proven to be a very useful tool for achieving coordination in the distributed systems and aligning incentives of various actors. RedStone token facilitates data sharing ecosystem incentivising participants to produce, publish and validate data in a continuous and diligent way.
+
+##### Data access fees
+The end users who benefit from access to valuable information use tokens to reward providers that published these data. The exact fee and the subscription terms are at the discretion of the provider and depend on their effort, demand for data and potential competition.
+
+##### Staking
+Every provider needs to publish a Service Level Agreement describing the scope of data being served, the source of information, and the frequency of updates. In case a provider breach the terms of service, there will be a penalty applied which is also denominated in tokens. In order to assure users that any further claims could be fully covered, providers need to put aside a certain amount of token and lock it for a period of time. These funds are labelled as a stake in the ecosystem and are an important factor for users to select the most reliable provider. 
+
+##### Dispute resolution
+Because of the diverse nature of provided information, it will not always be possible to decide if a data was corrupted. Therefore, it will be necessary to have fallback procedure to resolve any disputes about data quality. The process could be facilitated by tokens when juries will be rewarded for voting alongside the majority and punished for supporting a losing side.
+
+##### Bootstrapping market
+At the early stage of development, the token could be distributed to providers to reward their availability and bootstrap the market before there is enough demand coming for data users.
+
+![redstone token design](img/redstone-token-design.png)
+
 ## RedStone Node
 ### General description (from Readme)
 ### Node architecture

docs/DISPUTE_RESOLUTION.md

@@ -9,7 +9,7 @@ The majority rule is the most common strategy for reaching coordination in a dec
 
 One of the adaptations of this idea was drafted by Vitalik Buterin in 2014 as a Schelling Coin concept. It was also proposed as an architecture for the Ethereum Price Feeds: “For financial contracts for difference, it may actually be possible to decentralize the data feed via a protocol called SchellingCoin” (from the Ethereum whitepaper). However, the network congestion, high gas price and the extreme storage cost of the Ethereum Virtual Machine rendered the solution impractical at the current level of technology.
 
-New generations of blockchains like Arweave could finally make the implementation of Schelling-point based algorithms economically feasible due to the cheaper storage and low contracts execution cost. The challenging part is setting the system parameters to keep the user friction as low as possible and reduce the voting requirement voting while maintaining the high quality of decisions. Hopefully, the recent simulations done by the Limestone team prove that even a small percentage of expert jurors could produce high-quality results when the incentives are properly set up (see the simulations of TCR-based decision model presented during EdCon Paris 2019).
+New generations of blockchains like Arweave could finally make the implementation of Schelling-point based algorithms economically feasible due to the cheaper storage and low contracts execution cost. The challenging part is setting the system parameters to keep the user friction as low as possible and reduce the voting requirement voting while maintaining the high quality of decisions. Hopefully, the recent simulations done by the Redstone team prove that even a small percentage of expert jurors could produce high-quality results when the incentives are properly set up (see the simulations of TCR-based decision model presented during EdCon Paris 2019).
 
 ## Dispute process
 The dispute is a multi-stage process that involves two main counter-parties: a challenger, who initiates the dispute, a provider, who originated the challenged data entry and a jury panel that makes the final decision. The actors are incentivised to participate with potential token rewards. All of the stages of the process are described below:
@@ -66,13 +66,13 @@ The diagram below presents smart-contracts with their connections and descriptio
 ## Challenges
 
 ### Low turnout
-Many blockchain voting systems suffer from voters’ apathy as governance proposals cannot attract the required quorum. Limestone mitigates this problem by introducing a gradual voting scheme. For the first voting on a dispute, there is a low quorum that increases with every appeal. This allows to resolve uncomplicated cases with little friction but allows for proper verification and majority voting if the case is complex enough.
+Many blockchain voting systems suffer from voters’ apathy as governance proposals cannot attract the required quorum. Redstone mitigates this problem by introducing a gradual voting scheme. For the first voting on a dispute, there is a low quorum that increases with every appeal. This allows to resolve uncomplicated cases with little friction but allows for proper verification and majority voting if the case is complex enough.
 
 ### Vote buying
-Decentralised voting systems use tokens to express voting power. Although the freedom to trade is a necessary feature of every token, buying a large amount of tokens just before important voting could derail the entire system. Not addressing this point was one of the most criticised vulnerabilities of the Aragon court. Limestone mitigates the issue by introducing another dimension to token ownership called holding capacity. It represents the maximum amount of tokens a juror could hold that can increase only as a result of making correct decisions. This ensures that jurors could gradually earn their voting power and the system is immune to short-term votes buying
+Decentralised voting systems use tokens to express voting power. Although the freedom to trade is a necessary feature of every token, buying a large amount of tokens just before important voting could derail the entire system. Not addressing this point was one of the most criticised vulnerabilities of the Aragon court. Redstone mitigates the issue by introducing another dimension to token ownership called holding capacity. It represents the maximum amount of tokens a juror could hold that can increase only as a result of making correct decisions. This ensures that jurors could gradually earn their voting power and the system is immune to short-term votes buying
 
 ### Jurors selection
-Jurors benefit from participating in the judgement, therefore there may be a tough competition to vote for a high-stake dispute. To prevent front-running and guarantee an equal right to vote for all the token holders, Limestone plans to implement a random-based selection process. The chances to be selected as a judge are proportional to the voting capacity, a variable that describes that increased gradually with every successful choice made by a juror. This ensures that the best arbiters will have the most opportunities to resolve disputes.
+Jurors benefit from participating in the judgement, therefore there may be a tough competition to vote for a high-stake dispute. To prevent front-running and guarantee an equal right to vote for all the token holders, Redstone plans to implement a random-based selection process. The chances to be selected as a judge are proportional to the voting capacity, a variable that describes that increased gradually with every successful choice made by a juror. This ensures that the best arbiters will have the most opportunities to resolve disputes.
 
 ### Privacy
 In most of the cases, public voting will be sufficient and the most cost-effective method to judge the dispute. However, in special cases involving high-stake or confidential content, there could be a need for a privacy-preserving process. We could easily extend the voting mechanism implemented by the Tribunal contract to support either a two-phase commit-reveal process or use zero-knowledge proofs of jurors’ decision.

docs/TOKEN_DESIGN.md

@@ -13,7 +13,7 @@ The token will be used to:
 ## Longer description
 
 ### Usage of the token
-Tokens are proven to be a very useful tool for achieving coordination in the distributed systems and aligning incentives of various actors. Limestone token facilitates data sharing ecosystem incentivising participants to produce, publish and validate data in a continuous and diligent way.
+Tokens are proven to be a very useful tool for achieving coordination in the distributed systems and aligning incentives of various actors. RedStone token facilitates data sharing ecosystem incentivising participants to produce, publish and validate data in a continuous and diligent way.
 
 #### Data access fees
 The end users who benefit from access to valuable information use tokens to reward providers that published these data. The exact fee and the subscription terms are at the discretion of the provider and depend on their effort, demand for data and potential competition.

tools/arweave/generate-tx-time-report.js

@@ -20,7 +20,7 @@ main();
 
 async function main() {
   const transactions = await getTransactionWithPagination({ maxPages: 50 });
-  // const { transactions } = await getLimestoneTransactions();
+  // const { transactions } = await getRedstoneTransactions();
 
   const minutesOfDelay = [], times = [];
 
@@ -65,7 +65,7 @@ async function getTransactionWithPagination({ maxPages }) {
 
   while (hasNextPage && pageNr < maxPages) {
     console.log(`Getting transactions from page nr ${pageNr}. Cursor: ${after}`);
-    const response = await getLimestoneTransactions(after);
+    const response = await getRedstoneTransactions(after);
     allTransactions = allTransactions.concat(response.transactions);
     after = _.last(response.transactions).cursor;
     hasNextPage = response.hasNextPage;
@@ -76,7 +76,7 @@ async function getTransactionWithPagination({ maxPages }) {
     tx.node.tags.find(t => t.name === "timestamp").value));
 }
 
-async function getLimestoneTransactions(after) {
+async function getRedstoneTransactions(after) {
   const networkInfo = await arweave.network.getInfo();
   const minBlock = networkInfo.height - LAST_BLOCKS_TO_CHECK;