30 Commits

Author SHA1 Message Date
Cezary Haliniarz
587cc12b73 chore: rename $YYAV3SA1 to YYAV3SA1 (#131) 2022-08-11 22:27:55 +02:00
Cezary Haliniarz
8a9aec61bc fix: update node runner integration test (#129) 2022-07-21 18:12:00 +02:00
Cezary Haliniarz
bb1b50b378 feat: add Avalanche YieldYak fetcher (#108)
* feat: add multicall
2022-07-21 16:44:11 +02:00
Cezary Haliniarz
25c1a92da2 feat: add oracle registry and manifest fallback (#103)
* feat: add oracle registry and manifest fallback

* feat: adjust to new promise-timeout version

* chore: refactor fetching oracle registry state
2022-07-08 16:12:13 +02:00
Cezary Haliniarz
4c37d38d0b fix: don't kill node when update manifest fails (#99)
* Introduce promise-timeout own implementation
2022-07-07 16:04:59 +02:00
Cezary Haliniarz
a8495e5a47 chore: apply prettier to all files (#116) 2022-07-07 15:33:05 +02:00
Cezary Haliniarz
c8e86cfc07 feat: move configuration to env variables (#95) 2022-07-07 11:28:25 +02:00
Cezary Haliniarz
43a5006180 chore: add more fields to oracle registry contract (#100)
* chore: add more fields to oracle registry contract

* Renames
2022-06-23 01:57:40 +02:00
Dmitrii Novikov
89bdfc2308 feat: implement twelve-data-fetcher (#93) 2022-06-09 21:17:36 +02:00
Alex Suvorov
299afa90aa feat: custom urls fetcher on demand (#85)
* request proxying implemented

* fix: error handling implemented

* fix: fixes after review

* chore: custom urls fetcher improved
2022-05-20 23:04:39 +02:00
Alex Suvorov
a4c73ee062 feat: request proxying implemented (#74)
* request proxying implemented

* fix: error handling implemented

* fix: fixes after review
2022-05-20 19:02:09 +02:00
Alex Suvorov
2c0742093e fix: sources cleaning and refactoring (#71)
* bundlr integration code added

* added bundlr and arweave scripts

* bundlr integrated, arweave signing and tx posting removed

* few fixes after tests

* tests fixed

* 0.4.25

* Hatskier/connecting to bundlr (#41)

* Hatskier/streamr broadcaster (#36)

* initial structure for streamr broadcaster

* streamr proxy implemented

* tests fixed

* npm dependencies updates, docker versions upgraded, streamr building problem fixed

* fixes after ppe's review

* another try to fix tests

* Hatskier/running node in app runner (#38)

* experiment with built-in express app

* few fixes and improvements in Docker configuration and express server

* tests fixed

* tried t fixed github CI tests

* tests disabled temporary

* implemented a tool to analyze historical arweave fees

* streamr broadcaster improved

* Hatskier/drand fetcher (#39)

* drand fetcher implementation

* drand fetcher fixed

* sending values in package in http package broadcaster + added workflow for automated publishsing to ECR Public

* another try to run public ecr deployment

* dockefile path updated in workflow configuration

* updated ecr-public deployment workflow

* added support for omitSourcesInArweaveTx param

* docker-avalanche-2 tool created

* bugfix for sources cleaning

* swithed exitIfARBalanceLow to warnIfARBalanceLow on node start

* added SOS cryptocurrency support

* 0.4.22

* env config validation improved, added debug logging

* removed debug logging

* created a script for automated random config generation

* added 2 new providers

* 0.4.23

* 0.4.24

* bundlr integration code added

* added bundlr and arweave scripts

* bundlr integrated, arweave signing and tx posting removed

* few fixes after tests

* tests fixed

* 0.4.25

* added a new script: prepare-light-tokens-config-for-redstone-api.js

* fix: one failed broadcaster should not block another broadcaster

* removed unused import

* Hatskier/connecting to bundlr (#45)

* changed default source timeout to 15s

* changed default source timeout to 20s

* improved broadcasting for several http cache layers

* Registration update (#49)

* changed default source timeout to 15s

* changed default source timeout to 20s

* improved broadcasting for several http cache layers

* removed signature and replaced signer address with signer public key

* removed unused code

* redstone gateway connected, dockerfiles updated

* added price estimation script for bundlr

* Update providers registration

Co-authored-by: hatskier <suvorovalex1997@gmail.com>
Co-authored-by: c-gma <sigma@deltaprime.io>

* fix: removed streamr to test a hypothesis that it influenced nodes performance

* chore: added logging of helpful details (evm address, git commit)

* chore: added diagnostic info printing

* removed coinbase fetcher

* chore: data sources cleaning and fixing

* chore: ccxt fetchers refactoring

* chore: reverted dev.json manifest in sample-config.json

* Trigger docker update

* Update README.md

* chore: removed unused code and comments after review

Co-authored-by: Jakub Wojciechowski <jakub@alice.si>
Co-authored-by: c-gma <sigma@deltaprime.io>
2022-05-16 00:40:07 +02:00
Alex Suvorov
07235b8f63 Hatskier/custom urls fetcher (#68)
* bundlr integration code added

* added bundlr and arweave scripts

* bundlr integrated, arweave signing and tx posting removed

* few fixes after tests

* tests fixed

* 0.4.25

* added a new script: prepare-light-tokens-config-for-redstone-api.js

* fix: one failed broadcaster should not block another broadcaster

* removed unused import

* changed default source timeout to 15s

* changed default source timeout to 20s

* improved broadcasting for several http cache layers

* removed signature and replaced signer address with signer public key

* removed unused code

* redstone gateway connected, dockerfiles updated

* added price estimation script for bundlr

* config generation improved

* error logging improved and added sample twaps manifest

* draft twap implementation added

* added mechanism to disable arweave backup

* twap calculation logic implemented

* updated TwapFetcher structure to support signature verification

* few fixes in twap fetcher

* added CELO and AVAX tokens to rapid manifest

* twap fetcher implementation completed

* added a script to print all available fetchers

* added tests for twap fetcher

* Edge cases

* minor updates

* edge cases fixes

* added symbol to id mapping in base fetcher implementation

* added more symbols and sources

* added required files for manifest generation

* updated symbols config

* removed unused ARS symbol

* 0.4.26

* fixed yf-unofficial tests

* 0.4.27

* added private dockerfiles to gitignore

* added example manifest for custom urls

* added PoC for custom-urls fetcher

* 0.4.28

* updated custom-urls PoC

* customUrlsDetails -> customUrlDetails

* 0.4.29

* reverted signer address verification

* 0.4.30

Co-authored-by: c-gma <sigma@deltaprime.io>
2022-05-09 17:50:11 +02:00
Alex Suvorov
767ae7c662 feat: connecting to bundlr network (#40)
* bundlr integration code added

* added bundlr and arweave scripts

* bundlr integrated, arweave signing and tx posting removed

* few fixes after tests

* tests fixed

* 0.4.25

* added a new script: prepare-light-tokens-config-for-redstone-api.js

* fix: one failed broadcaster should not block another broadcaster

* removed unused import

* changed default source timeout to 15s

* changed default source timeout to 20s

* improved broadcasting for several http cache layers

* removed signature and replaced signer address with signer public key

* removed unused code

* redstone gateway connected, dockerfiles updated

* added price estimation script for bundlr

* feat: twaps (#50)

* config generation improved

* error logging improved and added sample twaps manifest

* draft twap implementation added

* added mechanism to disable arweave backup

* twap calculation logic implemented

* updated TwapFetcher structure to support signature verification

* few fixes in twap fetcher

* added CELO and AVAX tokens to rapid manifest

* twap fetcher implementation completed

* added a script to print all available fetchers

* added tests for twap fetcher

* Edge cases

* minor updates

* edge cases fixes

Co-authored-by: c-gma <sigma@deltaprime.io>

* feat: twaps (#66)

* config generation improved

* error logging improved and added sample twaps manifest

* draft twap implementation added

* added mechanism to disable arweave backup

* twap calculation logic implemented

* updated TwapFetcher structure to support signature verification

* few fixes in twap fetcher

* added CELO and AVAX tokens to rapid manifest

* twap fetcher implementation completed

* added a script to print all available fetchers

* added tests for twap fetcher

* Edge cases

* minor updates

* edge cases fixes

* feat: symbol to id mapping (#52)

* added symbol to id mapping in base fetcher implementation

* added more symbols and sources

* added required files for manifest generation

* updated symbols config

* removed unused ARS symbol

* 0.4.26

* fixed yf-unofficial tests

* 0.4.27

* added private dockerfiles to gitignore

Co-authored-by: c-gma <sigma@deltaprime.io>

Co-authored-by: c-gma <sigma@deltaprime.io>
2022-04-28 00:10:49 +02:00
Cezary Haliniarz
c55559eb7b feat: add redstone oracle registry (#51) 2022-04-20 09:32:21 +02:00
hatskier
ef56a36121 swithed exitIfARBalanceLow to warnIfARBalanceLow on node start 2022-01-11 13:29:53 +01:00
Alex Suvorov
8dd07c3ffa Hatskier/streamr broadcaster (#36)
* initial structure for streamr broadcaster

* streamr proxy implemented

* tests fixed

* npm dependencies updates, docker versions upgraded, streamr building problem fixed

* fixes after ppe's review

* another try to fix tests

* Hatskier/running node in app runner (#38)

* experiment with built-in express app

* few fixes and improvements in Docker configuration and express server

* tests fixed

* tried t fixed github CI tests

* tests disabled temporary

* implemented a tool to analyze historical arweave fees

* streamr broadcaster improved

* Hatskier/drand fetcher (#39)

* drand fetcher implementation

* drand fetcher fixed

* sending values in package in http package broadcaster + added workflow for automated publishsing to ECR Public
2021-12-08 16:49:07 +01:00
Alex Suvorov
fd2f119787 Added some Latin American currencies (#33)
* Added some Latin American currencies (#31)

* api-dojo-rapid fetcher updated

* manifests updated

* minor improvement in the api-dojo-rapid-fetcher

Co-authored-by: DCota <32775237+DaigaroCota@users.noreply.github.com>
2021-11-09 16:36:00 +01:00
Alex Suvorov
2f5fa27a4d Hatskier/adding lite evm signature (#17)
* evm lite signer implemented

* lite signature generation connected to single prices and price packages

* tests implemented for lite evm signature

* manually added logos for verto tokens to predefined config

* tokens config updated

* getLiteDataBytesString and getDataToSign made public in EvmPriceSigner

* 0.4.5

* 0.4.6

* 0.4.7

* redstone-node version updated

* 0.4.10

* 0.4.11

* tsconfig updated to copy manifests to dist folder

* 0.4.12

* verto tag renamed to pst

* 0.4.13

* 0.4.14

* Hatskier/adding pangoling fetcher (#18)

* pangolin fetcher implementation

* tests implemented

* PST and pangolin token fixes

* pangolin added to sources config

* tokens config update intstructions added to README

* AVAX tokens added to config

* 0.4.15

* added avax tag to some avalanche tokens

* 0.4.16

* bad ccxt sources removed

* evm address added to providers.json

* 0.4.17

* unstable tokens removed

* pangolin manifests for different base tokens added

* main manifest cleaned up, added pangolin fetchers

* pangolin-aablock fetcher fixed

* tests fixed

* renamed all-supported-tokens.json to main.json

* implemented a tool for redstone-rapid manifest creation

* trader-joe fetcher implemented

* manifests updated for tokens from avalanche ecosystem

* sources config updated

* JOE and WAVAX tokens added to config

* tests added for DEXes

* 0.4.18

* added providers to some avalanche tokens

* 0.4.19

* ccxt version updated

* fixes after ppe's review
2021-09-06 09:32:36 +02:00
Alex Suvorov
ae78874756 Hatskier/ccxt fetchers (#14)
* ccxt fetcher implementation

* ccxt fetchers connected

* tests fixed

* verto refactoring, ccxt fetchers manifests updated

* ccxt tests implemented

* old fetchers refactoring started

* USDT fetching mocked in tests

* coingecko and coinbase fetchers refactoring

* ecb fetcher refactoring

* kyber fetchers refactoring

* uniswap and sushiswap fetchers refactoring

* yf-unofficial fetcher refactoring

* manifests updated for ccxt fetchers

* increased timeout for ccxt fetchers

* ccxt manifest generation script improved. Now it compares prices from ccxt fetchers  with existing redstone-api prices

* manifests updated

* BTMX token removed from coingecko source

* ccxt manifests updated

* tmp log file removed

* enableRateLimit set to false for ccxt fetchers

* increased maxPriceDeviation for unpopular tokens

* implemented script for sources-list generation

* fixes after review: fetcher files renaming

* fixes after review: replaced string type with ExchangeId for ccxt fetchers

* fixes after review: replaced lodash methods with built-in functions

* fixes after review: classes refactored

* fixes after review: const renamed MAX_RESPONSE_TIME_TO_RETRY_FETCHING_MS

* ccxt manifest generation script updated to support option of avoiding new tokens adding

* fixes after review: refactored fetching additional pricing data for USDT and ETH inside extractPrices method

* unstable sources removed

* aggregation mechanism improved, tests added

* added providers.json to redstone-node configs
2021-08-04 19:56:28 +02:00
piotruda
c49bc3df7d New fetcher - Verto (#11)
* Token list generator

* Token list generator - automation

* Token list generator - code cleanup

* New fetcher - Verto

* Review corrections

Co-authored-by: Piotr Duda <piotrduda1992@gmail.com>
Co-authored-by: hatskier <suvorovalex1997@gmail.com>
2021-07-22 02:13:38 +02:00
Alex Suvorov
5aff5c93f3 Hatskier/api-dojo-rapid fetcher implementation (#12)
* fetchers refactoring started, yahoo-finance-rapid fetcher implemented

* fix in tests

* yahoo-finance fetcher renamed to api-dojo-rapid (#13)
2021-07-20 13:45:38 +02:00
Piotr Pędziwiatr
830ed3bdba call to SWC using SDK (#6) 2021-07-02 10:16:11 +02:00
Alex Suvorov
daaaef1073 EVM signer for single prices implemented (#7)
* evm signer for single prices implemented

* commented code removed

* minor fixes after Kuba's review and tests improvement
2021-07-02 00:56:35 +02:00
ppedziwiatr
35f73e6d42 read manifest from SmartWeave contract (#1)
* read manifest from SmartWeave contract

* non-blocking manifest update

* nodejs version update from redstone nodes
2021-06-17 11:21:56 +02:00
Alex Suvorov
a7ba194a2f Evm signature support (#4)
* EvmPriceSigner implemented

* simple tests implemented for EvmPriceSigner

* added comments for further test cases

* tests improved for EcmPriceSigner

* tests fixed

* evm signature implementation

* tests fixes and improvements

* fixes after Kuba's review

* switched back to arweave.net
2021-06-15 15:35:46 +02:00
Alex Suvorov
6c3d7ffc98 Barchart fetcher implementation (#2)
* barchart fetcher implemented

* barchart fetcher tests implemented

* added commdities and indices support to barchart fetcher

* barchart invalid response handling improved

* DIA symbol removed
2021-06-10 15:02:29 +02:00
Alex Suvorov
d5476082ac Warnings and errors fix, performance tracking fixed (#3)
* temporary fixes for trackEnd problems debugging

* promise timeout fixes

* performance tracking refactoring

* balancer fetcher removed, log levels updated

* fixes after review
2021-06-10 14:56:29 +02:00
ppedziwiatr
9c411c9e8f check on minimumArBalance config param added 2021-05-31 15:28:49 +02:00
ppedziwiatr
17883f7815 code migration 2021-05-31 14:03:28 +02:00