29 Commits

Author SHA1 Message Date
Cezary Haliniarz
bb1b50b378 feat: add Avalanche YieldYak fetcher (#108)
* feat: add multicall
2022-07-21 16:44:11 +02:00
Cezary Haliniarz
25c1a92da2 feat: add oracle registry and manifest fallback (#103)
* feat: add oracle registry and manifest fallback

* feat: adjust to new promise-timeout version

* chore: refactor fetching oracle registry state
2022-07-08 16:12:13 +02:00
Cezary Haliniarz
4c37d38d0b fix: don't kill node when update manifest fails (#99)
* Introduce promise-timeout own implementation
2022-07-07 16:04:59 +02:00
Cezary Haliniarz
c8e86cfc07 feat: move configuration to env variables (#95) 2022-07-07 11:28:25 +02:00
Cezary Haliniarz
b62dafef7f feat: add coinbase to ccxt fetcher (#96)
* feat: add coinbase to ccxt fetcher

* Remove plain coin from ccxt

* Remove coinbase fetcher remainings

* Repopulate main manifest
2022-06-23 01:56:04 +02:00
Cezary Haliniarz
abba13e464 fix: data issues after check 01.06.2022 (#94) 2022-06-14 15:32:19 +02:00
Dmitrii Novikov
89bdfc2308 feat: implement twelve-data-fetcher (#93) 2022-06-09 21:17:36 +02:00
hatskier
3c184833f1 chore: removed ftx from glmr symbol 2022-06-01 14:52:59 +02:00
Alex Suvorov
a4c73ee062 feat: request proxying implemented (#74)
* request proxying implemented

* fix: error handling implemented

* fix: fixes after review
2022-05-20 19:02:09 +02:00
Alex Suvorov
2c0742093e fix: sources cleaning and refactoring (#71)
* bundlr integration code added

* added bundlr and arweave scripts

* bundlr integrated, arweave signing and tx posting removed

* few fixes after tests

* tests fixed

* 0.4.25

* Hatskier/connecting to bundlr (#41)

* Hatskier/streamr broadcaster (#36)

* initial structure for streamr broadcaster

* streamr proxy implemented

* tests fixed

* npm dependencies updates, docker versions upgraded, streamr building problem fixed

* fixes after ppe's review

* another try to fix tests

* Hatskier/running node in app runner (#38)

* experiment with built-in express app

* few fixes and improvements in Docker configuration and express server

* tests fixed

* tried t fixed github CI tests

* tests disabled temporary

* implemented a tool to analyze historical arweave fees

* streamr broadcaster improved

* Hatskier/drand fetcher (#39)

* drand fetcher implementation

* drand fetcher fixed

* sending values in package in http package broadcaster + added workflow for automated publishsing to ECR Public

* another try to run public ecr deployment

* dockefile path updated in workflow configuration

* updated ecr-public deployment workflow

* added support for omitSourcesInArweaveTx param

* docker-avalanche-2 tool created

* bugfix for sources cleaning

* swithed exitIfARBalanceLow to warnIfARBalanceLow on node start

* added SOS cryptocurrency support

* 0.4.22

* env config validation improved, added debug logging

* removed debug logging

* created a script for automated random config generation

* added 2 new providers

* 0.4.23

* 0.4.24

* bundlr integration code added

* added bundlr and arweave scripts

* bundlr integrated, arweave signing and tx posting removed

* few fixes after tests

* tests fixed

* 0.4.25

* added a new script: prepare-light-tokens-config-for-redstone-api.js

* fix: one failed broadcaster should not block another broadcaster

* removed unused import

* Hatskier/connecting to bundlr (#45)

* changed default source timeout to 15s

* changed default source timeout to 20s

* improved broadcasting for several http cache layers

* Registration update (#49)

* changed default source timeout to 15s

* changed default source timeout to 20s

* improved broadcasting for several http cache layers

* removed signature and replaced signer address with signer public key

* removed unused code

* redstone gateway connected, dockerfiles updated

* added price estimation script for bundlr

* Update providers registration

Co-authored-by: hatskier <suvorovalex1997@gmail.com>
Co-authored-by: c-gma <sigma@deltaprime.io>

* fix: removed streamr to test a hypothesis that it influenced nodes performance

* chore: added logging of helpful details (evm address, git commit)

* chore: added diagnostic info printing

* removed coinbase fetcher

* chore: data sources cleaning and fixing

* chore: ccxt fetchers refactoring

* chore: reverted dev.json manifest in sample-config.json

* Trigger docker update

* Update README.md

* chore: removed unused code and comments after review

Co-authored-by: Jakub Wojciechowski <jakub@alice.si>
Co-authored-by: c-gma <sigma@deltaprime.io>
2022-05-16 00:40:07 +02:00
hatskier
995e8e686a chore: bundlr-client updated 2022-05-15 18:39:42 +02:00
Alex Suvorov
07235b8f63 Hatskier/custom urls fetcher (#68)
* bundlr integration code added

* added bundlr and arweave scripts

* bundlr integrated, arweave signing and tx posting removed

* few fixes after tests

* tests fixed

* 0.4.25

* added a new script: prepare-light-tokens-config-for-redstone-api.js

* fix: one failed broadcaster should not block another broadcaster

* removed unused import

* changed default source timeout to 15s

* changed default source timeout to 20s

* improved broadcasting for several http cache layers

* removed signature and replaced signer address with signer public key

* removed unused code

* redstone gateway connected, dockerfiles updated

* added price estimation script for bundlr

* config generation improved

* error logging improved and added sample twaps manifest

* draft twap implementation added

* added mechanism to disable arweave backup

* twap calculation logic implemented

* updated TwapFetcher structure to support signature verification

* few fixes in twap fetcher

* added CELO and AVAX tokens to rapid manifest

* twap fetcher implementation completed

* added a script to print all available fetchers

* added tests for twap fetcher

* Edge cases

* minor updates

* edge cases fixes

* added symbol to id mapping in base fetcher implementation

* added more symbols and sources

* added required files for manifest generation

* updated symbols config

* removed unused ARS symbol

* 0.4.26

* fixed yf-unofficial tests

* 0.4.27

* added private dockerfiles to gitignore

* added example manifest for custom urls

* added PoC for custom-urls fetcher

* 0.4.28

* updated custom-urls PoC

* customUrlsDetails -> customUrlDetails

* 0.4.29

* reverted signer address verification

* 0.4.30

Co-authored-by: c-gma <sigma@deltaprime.io>
2022-05-09 17:50:11 +02:00
Alex Suvorov
767ae7c662 feat: connecting to bundlr network (#40)
* bundlr integration code added

* added bundlr and arweave scripts

* bundlr integrated, arweave signing and tx posting removed

* few fixes after tests

* tests fixed

* 0.4.25

* added a new script: prepare-light-tokens-config-for-redstone-api.js

* fix: one failed broadcaster should not block another broadcaster

* removed unused import

* changed default source timeout to 15s

* changed default source timeout to 20s

* improved broadcasting for several http cache layers

* removed signature and replaced signer address with signer public key

* removed unused code

* redstone gateway connected, dockerfiles updated

* added price estimation script for bundlr

* feat: twaps (#50)

* config generation improved

* error logging improved and added sample twaps manifest

* draft twap implementation added

* added mechanism to disable arweave backup

* twap calculation logic implemented

* updated TwapFetcher structure to support signature verification

* few fixes in twap fetcher

* added CELO and AVAX tokens to rapid manifest

* twap fetcher implementation completed

* added a script to print all available fetchers

* added tests for twap fetcher

* Edge cases

* minor updates

* edge cases fixes

Co-authored-by: c-gma <sigma@deltaprime.io>

* feat: twaps (#66)

* config generation improved

* error logging improved and added sample twaps manifest

* draft twap implementation added

* added mechanism to disable arweave backup

* twap calculation logic implemented

* updated TwapFetcher structure to support signature verification

* few fixes in twap fetcher

* added CELO and AVAX tokens to rapid manifest

* twap fetcher implementation completed

* added a script to print all available fetchers

* added tests for twap fetcher

* Edge cases

* minor updates

* edge cases fixes

* feat: symbol to id mapping (#52)

* added symbol to id mapping in base fetcher implementation

* added more symbols and sources

* added required files for manifest generation

* updated symbols config

* removed unused ARS symbol

* 0.4.26

* fixed yf-unofficial tests

* 0.4.27

* added private dockerfiles to gitignore

Co-authored-by: c-gma <sigma@deltaprime.io>

Co-authored-by: c-gma <sigma@deltaprime.io>
2022-04-28 00:10:49 +02:00
Cezary Haliniarz
6de30f3808 feat: add oracle contract scripts and cli (#57) 2022-04-22 10:49:23 +02:00
Cezary Haliniarz
c7a0975429 chore: update yarn.lock (#56) 2022-04-22 10:23:56 +02:00
Cezary Haliniarz
c55559eb7b feat: add redstone oracle registry (#51) 2022-04-20 09:32:21 +02:00
Alex Suvorov
8dd07c3ffa Hatskier/streamr broadcaster (#36)
* initial structure for streamr broadcaster

* streamr proxy implemented

* tests fixed

* npm dependencies updates, docker versions upgraded, streamr building problem fixed

* fixes after ppe's review

* another try to fix tests

* Hatskier/running node in app runner (#38)

* experiment with built-in express app

* few fixes and improvements in Docker configuration and express server

* tests fixed

* tried t fixed github CI tests

* tests disabled temporary

* implemented a tool to analyze historical arweave fees

* streamr broadcaster improved

* Hatskier/drand fetcher (#39)

* drand fetcher implementation

* drand fetcher fixed

* sending values in package in http package broadcaster + added workflow for automated publishsing to ECR Public
2021-12-08 16:49:07 +01:00
Alex Suvorov
c5529142f6 Hatskier/running avalanche node (#32)
* initial implementation of multi-broadcasting support

* public Dockerfile added, passing config through environment variables initial implementation

* generated manifest for redstone-avalanche provider, improved tools for manifest generation

* added docker:build command to package.json

* docker script updated

* avalanche manifest updated

* VRT token address updated

* coingecko token config updated

* added qi and yak tokens to config

* ccxt updated, implemented more fetchers for yak, qi and frax tokens

* tokens config updated (added redstone-avalanche provider)
2021-11-09 01:04:14 +01:00
Alex Suvorov
09e68f0f97 redstone-smartweave connected (#20)
* redstone-smartweave connected

* removed unused code
2021-09-17 12:59:15 +02:00
Alex Suvorov
2f5fa27a4d Hatskier/adding lite evm signature (#17)
* evm lite signer implemented

* lite signature generation connected to single prices and price packages

* tests implemented for lite evm signature

* manually added logos for verto tokens to predefined config

* tokens config updated

* getLiteDataBytesString and getDataToSign made public in EvmPriceSigner

* 0.4.5

* 0.4.6

* 0.4.7

* redstone-node version updated

* 0.4.10

* 0.4.11

* tsconfig updated to copy manifests to dist folder

* 0.4.12

* verto tag renamed to pst

* 0.4.13

* 0.4.14

* Hatskier/adding pangoling fetcher (#18)

* pangolin fetcher implementation

* tests implemented

* PST and pangolin token fixes

* pangolin added to sources config

* tokens config update intstructions added to README

* AVAX tokens added to config

* 0.4.15

* added avax tag to some avalanche tokens

* 0.4.16

* bad ccxt sources removed

* evm address added to providers.json

* 0.4.17

* unstable tokens removed

* pangolin manifests for different base tokens added

* main manifest cleaned up, added pangolin fetchers

* pangolin-aablock fetcher fixed

* tests fixed

* renamed all-supported-tokens.json to main.json

* implemented a tool for redstone-rapid manifest creation

* trader-joe fetcher implemented

* manifests updated for tokens from avalanche ecosystem

* sources config updated

* JOE and WAVAX tokens added to config

* tests added for DEXes

* 0.4.18

* added providers to some avalanche tokens

* 0.4.19

* ccxt version updated

* fixes after ppe's review
2021-09-06 09:32:36 +02:00
Alex Suvorov
8d79951a35 Hatskier/config generation scripts (#16)
* script for config generation created

* implemented scripts for token list generation for new fetchers

* unused files removed

* script structure created for token config generation script

* script for token config generation created

* updated config scripts to use source logos from redstone cdn

* created scripts for cdn images, updated tokens config

* added more logo images for tokens

* token logos published to redstone cdn

* donwload-symbol-logos script improved

* added 100+ new tokens from coingecko source

* manifest for redstone-rapid updated
2021-08-11 14:33:13 +02:00
Alex Suvorov
ae78874756 Hatskier/ccxt fetchers (#14)
* ccxt fetcher implementation

* ccxt fetchers connected

* tests fixed

* verto refactoring, ccxt fetchers manifests updated

* ccxt tests implemented

* old fetchers refactoring started

* USDT fetching mocked in tests

* coingecko and coinbase fetchers refactoring

* ecb fetcher refactoring

* kyber fetchers refactoring

* uniswap and sushiswap fetchers refactoring

* yf-unofficial fetcher refactoring

* manifests updated for ccxt fetchers

* increased timeout for ccxt fetchers

* ccxt manifest generation script improved. Now it compares prices from ccxt fetchers  with existing redstone-api prices

* manifests updated

* BTMX token removed from coingecko source

* ccxt manifests updated

* tmp log file removed

* enableRateLimit set to false for ccxt fetchers

* increased maxPriceDeviation for unpopular tokens

* implemented script for sources-list generation

* fixes after review: fetcher files renaming

* fixes after review: replaced string type with ExchangeId for ccxt fetchers

* fixes after review: replaced lodash methods with built-in functions

* fixes after review: classes refactored

* fixes after review: const renamed MAX_RESPONSE_TIME_TO_RETRY_FETCHING_MS

* ccxt manifest generation script updated to support option of avoiding new tokens adding

* fixes after review: refactored fetching additional pricing data for USDT and ETH inside extractPrices method

* unstable sources removed

* aggregation mechanism improved, tests added

* added providers.json to redstone-node configs
2021-08-04 19:56:28 +02:00
hatskier
f26b72166a arweave-multihost updated to version with 5xx codes support 2021-07-13 02:47:36 +02:00
Piotr Pędziwiatr
830ed3bdba call to SWC using SDK (#6) 2021-07-02 10:16:11 +02:00
hatskier
442d251925 arweave-multihost logging configuration 2021-06-26 18:26:43 +02:00
ppedziwiatr
971b329270 ArweaveMultihost added 2021-06-25 09:06:27 +02:00
ppedziwiatr
35f73e6d42 read manifest from SmartWeave contract (#1)
* read manifest from SmartWeave contract

* non-blocking manifest update

* nodejs version update from redstone nodes
2021-06-17 11:21:56 +02:00
Alex Suvorov
a7ba194a2f Evm signature support (#4)
* EvmPriceSigner implemented

* simple tests implemented for EvmPriceSigner

* added comments for further test cases

* tests improved for EcmPriceSigner

* tests fixed

* evm signature implementation

* tests fixes and improvements

* fixes after Kuba's review

* switched back to arweave.net
2021-06-15 15:35:46 +02:00
ppedziwiatr
17883f7815 code migration 2021-05-31 14:03:28 +02:00