212 Commits

Author SHA1 Message Date
Rafael Cardenas
033104bdcd chore: merge from master 2024-01-08 16:52:39 -06:00
Rafael Cárdenas
bf99e90fa5 fix: insert block transaction data in batches (#1760)
* chore: first pass of batching

* fix: tests

* refactor: rename bns tx header type

* chore: more refactors

* chore: upgrade toolkit

* chore: upgrade api toolkit

* fix: do not wait for pg notifications

* fix: parquet function calls

* fix: bns test imports

* fix: batch size

* fix: pox event writes

* chore: reorganize pox3 unlock height
2023-12-18 09:23:05 -06:00
Charlie
697895fe9a chore: package.json & package-lock.json to reduce vulnerabilities (#1771)
The following vulnerabilities are fixed with an upgrade:
- https://snyk.io/vuln/SNYK-JS-INFLIGHT-6095116

Co-authored-by: snyk-bot <snyk-bot@snyk.io>
2023-12-15 15:09:39 +01:00
Rafael Cárdenas
ceb7be08da feat: add /extended/v2/blocks endpoint with burn block filters (#1769)
* chore: add typebox

* feat: add blocks v2 endpoint

* chore: add types

* fix: v2 query

* fix: tests

* chore: tweaks

* docs: openapi

* fix: exports

* fix: tests

* fix: tokens router

* fix: adjust burn block hash regex
2023-12-14 09:38:29 -06:00
Matthew Little
2afb65cbb8 feat: update to latest TenureChange tx payload (#1767) 2023-12-13 18:32:55 +01:00
Matthew Little
285806f46c feat: pox-4 support (#1754)
* feat: handle pox-3 force unlocks

* chore: todo note

* feat: handle pox-3 force unlocks, pox-4 events, and revoke-delegate-stx event

* chore: rename pox-2 file

* chore: simplified pox route paths

* feat: add migration to create pox4_events table

* test: follow redirects in test fetch helper

* chore: fix storing different pox version events

* test: fix tests with missing pox4_events

* chore: move delegations lookup endpoint into pox router

* fix: handle pox4_events during reorgs

* test: fix pox4events reorg count

* chore: update stacks-node image to stacks 3.0

* test: begin transitioning pox tests to use pox-4

* chore: bump stacks-node to wip nakamoto branch

* test: switch more tests from pox-3 to pox-4

* fix: tx fee fall back in faucet

* test: update delegation check endpoints

* chore: isolate error in delegate-revoke test

* chore: pox3 to pox4 misc renames

* ci: rename 2.4 to 2.5

* test: remove no-longer applicable delegate-stx while stacking test

* chore: remove incorrect commend in pox4 test
2023-12-13 12:58:37 +01:00
Matthew Little
7c45f53622 feat: ingestion for TenureChange and NakamotoCoinbase tx types (#1753)
* feat: ingestion for TenureChange and NakamotoCoinbase tx types

* fix: add new tx fields to batch tx insert query

* chore: bump stacks-encoding-native-js

* test: fix flaky socket-io timeout test
2023-12-08 12:39:52 +01:00
Rafael Cárdenas
e192a6c0e2 refactor: remove old token metadata processor (#1750) 2023-11-14 09:45:05 -06:00
Rafael Cárdenas
a490d673f8 refactor: remove old token metadata processor (#1747)
* refactor: remove old processor, start fetching data from contract

* refactor: remove old tests

* fix: tests and lint

* fix: use lru cache

* chore: delete old tables

* chore: fix comments [skip ci]

* fix: env comment block [skip ci]

* chore: avoid duplicate RPC lookups

* fix: make an incremental migration to drop tables

* fix: re-add metadata processing ENV flag

---------

Co-authored-by: Matthew Little <zone117x@gmail.com>
2023-11-09 10:11:26 -06:00
Rafael Cárdenas
74606c7e47 refactor: use the API toolkit, upgrade to node 18 (#1743) 2023-10-31 13:05:35 -06:00
Chris Guimaraes
2d08e18760 working new_burn_block 2023-08-22 11:13:00 +01:00
Matthew Little
3dec5d1b62 Merge branch 'master' into develop
# Conflicts:
#	.env
#	.github/workflows/ci.yml
#	.vscode/launch.json
#	package.json
#	src/api/init.ts
#	src/datastore/common.ts
#	src/datastore/pg-store.ts
#	src/event-stream/event-server.ts
#	src/event-stream/reader.ts
#	src/index.ts
#	src/test-utils/test-helpers.ts
#	src/tests-2.4/pox-3-delegate-aggregation.ts
#	src/tests-bns-e2e/bns-integration-tests.ts
#	src/tests-bns/v1-import-tests.ts
#	src/tests-tokens-metadata/tokens-metadata-tests.ts
2023-07-06 13:51:39 +02:00
Chris Guimarães
14bb9f2c6d Logging migration to Pino library (#1630)
* feat: add new logger based on Pino library

* feat: changes on api/init.ts to use Pino logging library

* feat: changes on helpers.ts to use Pino logging library

* feat: changes on event-stream/event-server.ts to use Pino logging library

* feat: changes on API tests to use Pino logging library

* feat: changes on API codebase to use Pino logging library

* fix: lint

* feat: changes on logger

* feat: remove logError function

* feat: change logger.verbose to logger.debug

* fix: lint

* fix: first argument on logger should be an object

* feat: add logging information to ErrorRequestHandler

* fix: remove logger from ErrorRequestHandler

* fix: remove try-catch statement

Errors shoud be handled by the ErrorRequestHandler middleware

* feat: disable automatic 'request completed' logging
2023-05-03 17:25:46 +01:00
Matthew Little
5ba9bbf98b Merge branch 'master' into develop
# Conflicts:
#	CHANGELOG.md
#	src/datastore/pg-write-store.ts
2023-04-20 16:34:05 +02:00
Chris Guimarães
05d6a9e241 ci: npm publish on CI (#1610) 2023-04-12 17:15:13 +01:00
Matthew Little
a724b384a6 Merge branch 'master' into develop 2023-03-22 20:23:56 +01:00
CharlieC3
b56cc82144 ci: don't publish to npm for now 2023-03-22 14:58:07 -04:00
Nicholas Barnett
1f350ec45a feat: initial block download option to speed up chain sync (#1373)
* feat: add event-replay test suite

* test: bns genesis block

* chore: rename tsv to mainnet

* test: export import cycle

* chore: reduce mainnet tsv size

* fix(api): updates to event_observer_requests occur after specific event handlers

* fix: db has data check

* fix: drop views first, tables second

* feat: friendly error when migration cycle failed

* fix(api): clean up pr

* feat(ibd): added middleware for checking chainheight

* feat(ibd): need to query for chaintip each time, cannot rely on previous calculation

* feat(ibd): adding changes that will eventually have to be removed before pulling into master

* fix(events): added tests

* fix(event_observer_requests): modified env to run nodes

* feat(ibd): wrote a couple tests

* fix(duplicate-event-requests): fixed test

* fix(duplicate-event-requests): removed unneeded changes

* fix(duplicate-event-requests): reverted env

* feat(ibd): fixed tests and wrote another to test

* feat(ibd): cleaning pr

* fix(duplicate-events): addressing comments

* feat(duplicate-requests): address pr comments

* fix(duplicate-events): fixing tests

* feat(ibd): fixing tests

* fix(duplicate-events): fixing tests

* ci: security patch

* ci: patch

* feat(ibd-mode): fixing pr

* fix(duplicate-events): fixing pr

* fix(duplicate-events): fixing pr

* feat(ibd-mode): work in progress

* fix(duplicate-events): addressed pr feedback

* feat(ibd-plus-duplicate-events): prune mode activates ibd mode and added test

* feat(ibd-mode): rebased on develop

* feat: add event-replay test suite

* test: bns genesis block

* chore: rename tsv to mainnet

* test: export import cycle

* fix: db has data check

* fix: drop views first, tables second

* feat: friendly error when migration cycle failed

* feat(ibd-mode): fixing pr

* fix(events): added tests

* fix(duplicate-event-requests): removed unneeded changes

* fix(duplicate-events): fixing tests

* fix(duplicate-events): fixing pr

* fix(duplicate-events): addressed pr feedback

* feat(ibd): fixing pr

* feat(ibd): fixing pr

* feat(ibd-mode): re-add poisonblock handler

* feat: adding specific timeout for bns test

* feat: adding specific timeout for bns test

* feat: adding specific timeout for bns test

* feat: adding specific timeout for bns test

* feat: increase test timeout for bns

* feat: increase test timeout for bns

* feat(ibd): clean pr by removing unused file and comments

* feat(ibd-mode): fixed tests

* feat(ibd): reducing size of tsv files to make the tests run faster

---------

Co-authored-by: Rafael Cárdenas <rafael@rafaelcr.com>
2023-03-02 22:25:35 -06:00
Matthew Little
5d7056c1ba feat: support for subnets (#1549)
* test: initial integration test for subnets

* test: ensure first subnet block is mined

* feat: synthetic nft deposit tx

* test: increase timeout for subnets test

* test: attempt fix test flakiness in CI

* test: much faster subnet block mining time, subnet tests 5-10x faster

* ci: bump subnet-node docker image

* test: complete withdrawal on stacks chain

* test: try waiting for subnet contract interface to be ready

* test: verify NFT ownership on L1 after withdrawal

* test: fix subnet node port env vars

* feat: implement parsing for FT deposit synthetic tx

* chore: rename test file

* chore: do not skip test

* test: progress on subnet STX use-case test

* feat: implement parsing for STX deposit synthetic tx

* test: validation of synthetic tx for L1 deposit-stx

* test: validation of synthetic tx for L1 deposit-nft-asset

* test: validation of synthetic tx for L1 deposit-ft-asset

* chore: lint fix

* chore: remove debug comments
2023-02-14 13:30:41 +01:00
Matthew Little
5a5f07c6d3 chore: bump bitcoinjs-lib to latest (fixes warnings related to taproot/segwit_v1 addresses) (#1517)
* chore: bump to bitcoinjs-lib to latest (fixes warnings related to taproot/segwitV2 addresses)

* chore: bitcoinjs-lib now supports taproot addresses without custom handling
2023-01-11 17:19:34 +01:00
Matthew Little
dcbdfb9069 feat: Stacks 2.1 support (#1498)
* feat: support for Stacks 2.1 `versioned-smart-contract` transactions (#1279)

* dev: use latest `next` branch (Stacks 2.1) in docker-compose / dev environment

* chore: setup for testing with a local stacks-node v2.1 binary

* feat: support for versioned-smart-contract txs (Stacks 2.1)

* chore: update `stacks-encoding-native-js` dep to latest alpha

* fix: smart_contracts table missing the `clarity_version` column

* chore: test fixes

* chore: update test & dev env, and test suites to Stacks 2.1 image (#1280)

* fix: update several test suites to work with Stacks 2.1 docker image

* chore: move rosetta network endpoint tests to after more time-sensitive construction tests

* fix: attempt fix for `tests-rosetta-cli-data`

* chore: package.json fix

* fix: attempt fix for `tests-rosetta-cli-construction`

* chore: move test config into directory

* feat: separate suite for RPC request tests

* chore: update .github/workflows/ci.yml

Co-authored-by: Rafael Cárdenas <rafael@rafaelcr.com>

* chore: move btc faucet tests into own suite

* ci: add btc-faucet test to CI

Co-authored-by: Rafael Cárdenas <rafael@rafaelcr.com>

* chore: remove debug console.log

* chore: tests for versioned-smart-contact txs

* chore: update mocknet config to bootstrap to epoch 2.1

* chore: fix docker-compose background task logging in vscode debugger

* chore: remove local stacks-node debug and testing changes

Co-authored-by: Rafael Cárdenas <rafael@rafaelcr.com>

* feat: support for Stacks 2.1 `coinbase-pay-to-alt-recipient` txs (#1283)

* feat: support Stacks 2.1 `coinbase-pay-to-alt` tx types

* fix: add a `coinbase-to-alt` tx handling in a few more areas

* chore: fix coinbase in mempool table schema

* chore: update unit tests with new coinbase payload responses

* test: add tests for ingesting new coinbase tx payloads and generating expected API responses

* docs: better description for coinbase alt recipient

* feat: support for Stacks 2.1 stx-transfer event memo field (#1285)

* feat: support for Stacks 2.1 stx-transfer event memo field (the `stx-transfer-memo` Clarity2 function)

* fix: remove null memo field from different asset transfer events

* test: unit tests for Stacks 2.1 stx-transfer event memo field

* feat: include `stx-transfer-memo` results in `/address/<recipient>/stx_inbound` endpoint (#1289)

* feat: include Stacks 2.1 `stx-transfer-memo` in Rosetta token transfer operations (#1290)

* chore: cleanup Rosetta operation type strings

* feat: include `memo` from stx-transfer-memo events in Rosetta token transfer operations

* chore: lint fix

* chore: Rosetta tests for memo in stx-transfer-memo events

* fix: Rosetta memo strings should be decoded from hex

* chore: add `coinbase-pay-to-alt` tx type test for Rosetta parsing

* chore: upgrade to stacks.js v4.x (#1295)

* chore: upgrade to stacks.js v4.x

* chore: hardcode tx fees in rosetta tests

* chore: hardcode tx fees in token tests

* chore: hardcode tx fees in rosetta-cli-data tests

* chore: remove bugged fees in token tests

* chore: remove bugged fees in rosetta-cli-data tests

* chore: hardcode tx fees in rosetta-cli-data tests

* chore: fixes for bns tests

* feat: Stacks 2.1 PoX-2 support in Rosetta (#1339)

* feat: working setup for local krypton stacks-node miner with regtest bitcoind and auto-mining script

* feat: detect when stx mining starts for faster krypton stacks-node startup

* feat: use stacsk2.1-transition image for krypton debug deployment

* feat: update `/extended/v1/debug/broadcast/stack` endpoint to work with PoX-2

* chore: use PoX address decoding code from the WIP @stacks/stacking lib

* chore: update Rosetta PoX address parsing function

* chore(rosetta): skip extra /v2/info request when constructing stack-stx operation in Stacks 2.1

* feat: option in debug endpoint to use Rosetta to construct and submit Stacking transactions

* chore: simplify internal rosetta calls

* fix: Rosetta bug where parsing Stacking operations fails when the contract ABI is missing

* feat: Rosetta support for parsing PoX-2 contract calls

* feat: Rosetta support for parsing new Bitcoin address formats

* chore: remove no longer used testing configs and scripts

* chore: remove no longer used testing configs and scripts

* chore: debug page cleanup

* chore: configs for launching Stacks2.1-transition environment, and launching directly into Stacks2.1

* chore: task naming nit

* test: initial Stacks2.1 test suite

* chore: vscode debugger for 2.1 test suite

* ci: run 2.1 tests

* ci: possible fix for mapping host.docker.internal on linux

* chore: bitcoin address decoding without depending on try/catches for version detection

* chore: pin stacks2.1 images to hash

* chore: comment fix

* fix: report locked STX balance correctly when multiple event entries exist from stack-increase and stack-extend operations (#1363)

* chore: lint fixes

* chore: test moving migration files to non-ts source

* chore: rename migration file exts to js

* chore: fix pg migrations failing from certain jest scripts

* chore: cleanup jest.config.2.1.js

* chore: lint fix (revert)

* test: ensure locked stx balance event data has expected values after pox-2 stack-increase operation

* test: ensure stx unlock height event data has expected value after pox-2 stack-extend operation

* fix: report locked STX balance correctly when multiple event entries exist from stack-increase and stack-extend operations

* chore: refactor PoX-2 operation steps into separate tests

* chore: test scaffolding for pox-2 rewards

* chore: lint fixes

* test: validate Stacking slots and rewards after PoX-2 operations

* chore: update to bitcoin-v0.21.1 in stacks-2.1 test suite

* test: add additional 1 block padding to unlock height waiter for test flakiness

* chore(stacks-2.1): upgrade to bitcoinjs-lib-v6.x, initial P2TR work (#1367)

* chore: upgrade to bitcoinjs-lib-v6.x which has P2TR support

* feat: P2TR bitcoin address util

* chore: PoX-2 integration testing for segwit addresses (#1372)

* feat: functions for generating Bitcoin addresses in all formats supported by Stacks 2.1

* test: segwit PoX-2 integration tests

* test: e2e tests for PoX-2 segwit (P2WPKH) support

* chore: test PoX rewards against bitcoin json-rpc results

* test: e2e tests for PoX-2 rewards address formats p2pkh, p2sh, p2wpkh, p2wsh

* test: use more robust bitcoin `listtransactions` RPC method for validating PoX-2 rewards

* chore: update plain P2SH address generation to create a more standard P2SH(P2PKH) rather than 1-of-1 multisig address

* chore: temp skip segwit tests until `feat/native-segwit` stacks-blockchain branch is merged

* chore: Revert "temp skip segwit tests until `feat/native-segwit` stacks-blockchain branch is merged" -- too many other dependent changes

* chore: filter bitcoin rpc `listtransactions` result for only the address being tested

* ci: don't persist config in checkout

* feat: store Stacks block miner address value from new rewards payload introduced in https://github.com/stacks-network/stacks-blockchain/pull/3373 (#1413)

* chore: bump to latest Stacks 2.1 `next` branch (#1449)

* ci: configure semrel branches for 2.1 prerelease packages

* chore(release): 4.2.0-stacks-2.1.1 [skip ci]

# [4.2.0-stacks-2.1.1](https://github.com/hirosystems/stacks-blockchain-api/compare/v4.1.1...v4.2.0-stacks-2.1.1) (2022-11-22)

### Bug Fixes

* included query params in redirecting to prefix 0x in tx endpoint ([#1205](https://github.com/hirosystems/stacks-blockchain-api/issues/1205)) ([664cce7](664cce744d))
* incorrect websocket/socket.io transaction updates ([#1197](https://github.com/hirosystems/stacks-blockchain-api/issues/1197)) ([8ee1da8](8ee1da840b))
* mobx breakage by locking package dependencies ([#1206](https://github.com/hirosystems/stacks-blockchain-api/issues/1206)) ([5f8bc9f](5f8bc9fd4f))
* optimize `getMicroblocks` query ([#1179](https://github.com/hirosystems/stacks-blockchain-api/issues/1179)) ([7691109](769110926e))
* optimize block endpoint ([#1190](https://github.com/hirosystems/stacks-blockchain-api/issues/1190)) ([943e2d1](943e2d1c55))
* remove duplicate txs in microblock responses ([#1167](https://github.com/hirosystems/stacks-blockchain-api/issues/1167)) ([15c0c11](15c0c1124a))
* report locked STX balance correctly when multiple event entries exist from stack-increase and stack-extend operations ([#1363](https://github.com/hirosystems/stacks-blockchain-api/issues/1363)) ([d06beb3](d06beb32f1))
* upsert nft and ft metadata ([#1193](https://github.com/hirosystems/stacks-blockchain-api/issues/1193)) ([c4eec5d](c4eec5d060))

### Features

* add `transaction_count` for `microblocks_accepted` in block ([#1162](https://github.com/hirosystems/stacks-blockchain-api/issues/1162)) ([78d7d9c](78d7d9c9f7))
* add API version in response header ([#1216](https://github.com/hirosystems/stacks-blockchain-api/issues/1216)) ([1e998db](1e998db7d1))
* include `stx-transfer-memo` results in `/address/<recipient>/stx_inbound` endpoint ([#1289](https://github.com/hirosystems/stacks-blockchain-api/issues/1289)) ([cb5ca93](cb5ca933fe))
* include Stacks 2.1 `stx-transfer-memo` in Rosetta token transfer operations ([#1290](https://github.com/hirosystems/stacks-blockchain-api/issues/1290)) ([c7cc900](c7cc900edf))
* mempool stats endpoint and prometheus metrics ([#1241](https://github.com/hirosystems/stacks-blockchain-api/issues/1241)) ([9482238](9482238599))
* refactor pg classes, switch to postgres.js ([#1148](https://github.com/hirosystems/stacks-blockchain-api/issues/1148)) ([3ff4177](3ff41779f8)), closes [#1168](https://github.com/hirosystems/stacks-blockchain-api/issues/1168)
* send nft updates through websocket channels ([#1218](https://github.com/hirosystems/stacks-blockchain-api/issues/1218)) ([920a7b8](920a7b892a))
* Stacks 2.1 PoX-2 support in Rosetta ([#1339](https://github.com/hirosystems/stacks-blockchain-api/issues/1339)) ([e04bf64](e04bf64e6f))
* store Stacks block miner address value from new rewards payload introduced in https://github.com/stacks-network/stacks-blockchain/pull/3373 ([#1413](https://github.com/hirosystems/stacks-blockchain-api/issues/1413)) ([e34fab5](e34fab5e5d))
* support for Stacks 2.1 `coinbase-pay-to-alt-recipient` txs ([#1283](https://github.com/hirosystems/stacks-blockchain-api/issues/1283)) ([7e5752a](7e5752a7e6))
* support for Stacks 2.1 `versioned-smart-contract` transactions ([#1279](https://github.com/hirosystems/stacks-blockchain-api/issues/1279)) ([4ff1a91](4ff1a913ed)), closes [#1280](https://github.com/hirosystems/stacks-blockchain-api/issues/1280)
* support for Stacks 2.1 stx-transfer event memo field ([#1285](https://github.com/hirosystems/stacks-blockchain-api/issues/1285)) ([67332ce](67332ce699))

* feat: ingestion and querying for new PoX-2 events (#1403)

* test: Rosetta PoX-2 rewards and unlock events

* feat: progress on parsing and writing PoX2 events to postgres

* feat: sql insert for pox2 event data

* feat: reading pox2 event data from db, sql constraints on named pox event types

* feat: db-controller and initial GET route for reading & parsing pox2 events

* chore: progress on e2e testing for pox-2 delegation functionality

* chore: e2e testing for pox-2 delegation commit operation

* chore: e2e testing for pox-2 delegate-stack-increase

* chore: e2e testing for pox-2 delegate-stack-extend

* chore: e2e testing for pox-2 handle-unlock (missed reward slot)

* chore: lint fix, possible test fix

* chore: validation tests for each of the pox2-event payloads

* chore: potential test fixes

* chore: test fixes num 2

* chore: additional flaky test fixes

* feat: use pox2-events table to calculate account STX locked amount in /extended/v1/address balance endpoints

* chore: Rosetta integration tests for using segwit address in stacking operation

* test: ensure stacks RPC reports expected locked balance after Rosetta lock op

* chore: bump to latest pox2-events stacks-node image

* chore: move pox2-event parsing code into separate file

* ci: bump stacks2.1 test suite time to account for longer block mining interval from latest stacks-node changes

* chore: speed up tests with better "wait for pox cycle" logic

* ci: test fixes

* fix: adjustments to `handle-unlock` processing

* feat: add support for latest additions to pox2: `stack-aggregation-commit-indexed` and `stack-aggregation-increase`

* chore: integration tests for `stack-aggregation-commit-indexed` and `stack-aggregation-increase` pox2 event handling

* feat: store pox_v1_unlock_height as singleton in pg

* chore: initial setup for stacks-2.1 transition test suite

* chore: fix pox_state migrations

* feat: make account balance drop pox1-locked state after pox_v1_unlock_height is reached

* fix: rosetta block unlock events were limited to one account

* fix: off-by-one bug causing accounts to be reported as unlocked one block too early

* feat: add burn_block_height to db chainTip and /extended/status

* chore: fix datastore-tests from new status endpoint field

* chore: fix datastore-tests from new status endpoint field, second pass

* fix: adjust pox2_events query to handle special case with `handle-unlock`

* feat: add `contract_name` processing for the `stx_lock_event`, see https://github.com/stacks-network/stacks-blockchain/issues/3413

* chore: bump to latest https://github.com/stacks-network/stacks-blockchain/pull/3415

* chore: add account balance test for consistency between RPC and API balances throughout pox transition

* ci: matrix for stacks2.1-transition tests

* chore: start of Rosetta pox transition period tests

* chore: test Rosetta account balance endpoint in pox-2 stacking test suite

* ci: use unique account key for pox-2 rosetta test suite

* chore: test for pox_v1_unlock_height in Rosetta pox transition tests

* chore: refactor giant pox-2 test file into multiple files, run separately in ci

* feat: generate synthetic Rosetta stx unlock operations for the pox_v1_unlock_height block

* feat: generate Rosetta unlock operations from pox2 auto-unlock events

* fix: store locked balance from `handle-unlock` event rather than resetting to zero

* chore: remove todos

* chore: remove todos

* chore: remove no-longer correct sanity check

* chore: address PR feedback, composite index for pox2_events ordering

* fix: add default stx faucet tx fee if estimate not available (#1456)

* test: add integration test for stx faucet requests

* fix: add default stx faucet tx fee if estimate not available

* ci: disable generated doc type file test for now

* fix: add automatic pox switching to rosetta (#1454)

* refactor: add minor changes to rosetta

* chore: switch to stacks.js v6 and refactor helpers

* fix: update rosetta construction for 2.1

* test: refactor pox transition tests

* test: add btc address format rosetta test

* build: npm audit

* chore: lint fix json files not in src

* chore: fix exports

* test: add rosetta stacking for each block height

* test: add double stacking test

* chore: revert back to original suggested_fee_multiplier behavior

* chore: remove dynamic contract from dummy data

* chore: remove todos

* test: fix test address

* test: skip failing tests

* test: add check to rosetta contract tests

* chore: update to latest node next branch for testing

Co-authored-by: janniks <janniks@users.noreply.github.com>

* chore(release): 7.0.0-stacks-2.1.1 [skip ci]

# [7.0.0-stacks-2.1.1](https://github.com/hirosystems/stacks-blockchain-api/compare/v7.0.0-beta.1...v7.0.0-stacks-2.1.1) (2022-11-29)

### Bug Fixes

* add automatic pox switching to rosetta ([#1454](https://github.com/hirosystems/stacks-blockchain-api/issues/1454)) ([ad7e492](ad7e49216b))
* add default stx faucet tx fee if estimate not available ([#1456](https://github.com/hirosystems/stacks-blockchain-api/issues/1456)) ([eeeffd0](eeeffd0244))

### Features

* ingestion and querying for new PoX-2 events ([#1403](https://github.com/hirosystems/stacks-blockchain-api/issues/1403)) ([1936ba6](1936ba6519))

* ci: re-enable docs-lint

* chore: fix post-merge lint errors

* fix: use new `this.sqlTransaction(...)` in pox2 db queries

* ci: re-enable docs-lint

* chore: post-merge fixes

* ci: add missing gh job for `event-replay` test suite

* ci: increase timeout for 2.1 test suite

* chore: split up pox-2-rosetta tests (#1461)

* chore: split up pox-2-rosetta tests

* chore: split up pox-2-rosetta test suites into smaller units

* feat: [Stacks 2.1] Support new "block 0" boot events (#1476)

* chore: split up pox-2-rosetta tests

* chore: split up pox-2-rosetta test suites into smaller units

* feat: new regtest standalone image with 2.1 node

* chore: store postgres data in same volume

* chore: add configurable heights for each epoch

* feat: arg to toggle if docker image is bootstraps the chainstate

* chore: block height adjustments

* chore: block heights that work with epoch 2.0 and 2.05 receipts

* chore: fix bash script error

* feat: store and forward boot events

* chore: allow pg server connections

* chore: do not error when skipping bootstrap

* chore: update to latest stacks2.1 images

* chore: add epoch config for mocknet node

* ci: attempt fix for host.docker.internal for tests using stacks mocknet image

* ci: fix tokens and rosetta mocknet mode tests

* feat: finish ingesting "block 0" data once the first real block is received

* test: implement test suite to ensure "block-zero" is ingested correctly

* test: re-enable double-stacking test suite

* ci: add workflow dispatch for standalone docker image build [skip ci]

* ci: allow use of existing stacks-node binaries from docker context

* ci: allow stacks-api version tag to be set, require git sha args

* ci: add workflow dispatch for standalone docker image build [skip ci]

* chore: bump to latest stacks-blockchain `next` branch

* chore(release): 7.0.0-stacks-2.1.2 [skip ci]

## [7.0.0-stacks-2.1.2](https://github.com/hirosystems/stacks-blockchain-api/compare/v7.0.0-stacks-2.1.1...v7.0.0-stacks-2.1.2) (2022-12-21)

### ⚠ BREAKING CHANGES

* remove deprecated `/nft_events` endpoint (#1329)
* mark breaking change
* optimize tables and improve canonical treatment of BNS data (#1287)

### Features

* [Stacks 2.1] Support new "block 0" boot events ([#1476](https://github.com/hirosystems/stacks-blockchain-api/issues/1476)) ([3d1b8f6](3d1b8f6f86))
* add ENV configs for DB close and API shutdown timeouts ([#1366](https://github.com/hirosystems/stacks-blockchain-api/issues/1366)) ([444f008](444f008fe2))
* add indexes for index_block_hash on BNS tables ([#1304](https://github.com/hirosystems/stacks-blockchain-api/issues/1304)) ([bbf4b2d](bbf4b2d2b8))
* add owner index to subdomains table ([#1331](https://github.com/hirosystems/stacks-blockchain-api/issues/1331)) ([a6c5e12](a6c5e12faa))
* add token_type metadata for rosetta ft operations ([#1332](https://github.com/hirosystems/stacks-blockchain-api/issues/1332)) ([09af27b](09af27b24b))
* **agg-paging-limits:** aggregated all paging query limits ([#1401](https://github.com/hirosystems/stacks-blockchain-api/issues/1401)) ([0203d36](0203d36342)), closes [#1379](https://github.com/hirosystems/stacks-blockchain-api/issues/1379) [#1379](https://github.com/hirosystems/stacks-blockchain-api/issues/1379)
* configurable pg connection lifetime and idle timeouts ([#1355](https://github.com/hirosystems/stacks-blockchain-api/issues/1355)) ([46ccf06](46ccf0640d))
* mark breaking change ([669fd0d](669fd0d8c0))
* optimize tables and improve canonical treatment of BNS data ([#1287](https://github.com/hirosystems/stacks-blockchain-api/issues/1287)) ([1f64818](1f648187b8))

### Bug Fixes

* add memos to send-many-memo rosetta STX transfer operations ([#1389](https://github.com/hirosystems/stacks-blockchain-api/issues/1389)) ([0a552b8](0a552b8d8c))
* add owner index on subdomains table ([#1323](https://github.com/hirosystems/stacks-blockchain-api/issues/1323)) ([c9c6d05](c9c6d053fd))
* add postgres connection error checking for ECONNRESET code ([03a1896](03a1896cff))
* bump version ([3863cce](3863cce1a6))
* catch cache controller db errors ([#1368](https://github.com/hirosystems/stacks-blockchain-api/issues/1368)) ([f15df41](f15df41fa9))
* catch pg exceptions on queries outside of express ([#1348](https://github.com/hirosystems/stacks-blockchain-api/issues/1348)) ([1f07b85](1f07b8587c))
* consolidate db migrations ([#1314](https://github.com/hirosystems/stacks-blockchain-api/issues/1314)) ([d6bdf9f](d6bdf9faff))
* detect name transfers and renewals in special circumstances ([#1303](https://github.com/hirosystems/stacks-blockchain-api/issues/1303)) ([cd381a9](cd381a95b4))
* disable faucet endpoints on mainnet ([#1425](https://github.com/hirosystems/stacks-blockchain-api/issues/1425)) ([b79b9b4](b79b9b43d5))
* event_observer_requests json writes ([#1334](https://github.com/hirosystems/stacks-blockchain-api/issues/1334)) ([465aa0b](465aa0b42c))
* filter BNS processing for successful txs only ([#1309](https://github.com/hirosystems/stacks-blockchain-api/issues/1309)) ([6a12936](6a129369c6))
* get rosetta latest block from chain_tip view ([#1445](https://github.com/hirosystems/stacks-blockchain-api/issues/1445)) ([ad386d3](ad386d30d1))
* guarantee db is empty before performing a replay ([#1374](https://github.com/hirosystems/stacks-blockchain-api/issues/1374)) ([ef8e7a9](ef8e7a9185))
* guard against empty lists before querying postgres ([#1345](https://github.com/hirosystems/stacks-blockchain-api/issues/1345)) ([6c88a16](6c88a166c8))
* handle pg exceptions on web socket transmitter ([#1353](https://github.com/hirosystems/stacks-blockchain-api/issues/1353)) ([2e6448d](2e6448d7af))
* handle postgres dns lookup error ([#1433](https://github.com/hirosystems/stacks-blockchain-api/issues/1433)) ([e00efd4](e00efd4e64))
* handle postgres.js connection timeouts ([#1424](https://github.com/hirosystems/stacks-blockchain-api/issues/1424)) ([4a2a342](4a2a342b56))
* handle websocket messages with a priority queue ([#1427](https://github.com/hirosystems/stacks-blockchain-api/issues/1427)) ([f0cb01c](f0cb01c054))
* import BNS v1 data during event replay ([#1301](https://github.com/hirosystems/stacks-blockchain-api/issues/1301)) ([bc59817](bc59817aa9))
* lint docs ci dependencies ([#1458](https://github.com/hirosystems/stacks-blockchain-api/issues/1458)) ([90d0c7b](90d0c7b831))
* log PoisonMicroblock tx instead rather than throwing ([#1379](https://github.com/hirosystems/stacks-blockchain-api/issues/1379)) ([cee6352](cee63529b4))
* refresh materialized views concurrently ([#1270](https://github.com/hirosystems/stacks-blockchain-api/issues/1270)) ([057c541](057c541b8c))
* refresh materialized views concurrently in new pg format ([#1324](https://github.com/hirosystems/stacks-blockchain-api/issues/1324)) ([20b284f](20b284fa38))
* refresh materialized views in their own pg connection ([#1356](https://github.com/hirosystems/stacks-blockchain-api/issues/1356)) ([9433d3c](9433d3c9c2))
* remove duplicate tx socket updates inside db transactions ([#1360](https://github.com/hirosystems/stacks-blockchain-api/issues/1360)) ([60c185d](60c185d839))
* remove live tsv append ([#1315](https://github.com/hirosystems/stacks-blockchain-api/issues/1315)) ([e2a1247](e2a124710f))
* retry pg connection on new library code ([#1326](https://github.com/hirosystems/stacks-blockchain-api/issues/1326)) ([35db939](35db939199))
* revert to 404 error code on bns name errors ([#1440](https://github.com/hirosystems/stacks-blockchain-api/issues/1440)) ([cdc039c](cdc039cea8))
* skip migrations on read-only start ([#1351](https://github.com/hirosystems/stacks-blockchain-api/issues/1351)) ([1d32261](1d322614e7))
* sql transaction consistency ([#1410](https://github.com/hirosystems/stacks-blockchain-api/issues/1410)) ([01e26d9](01e26d9c89))
* support multiple BNS name events in the same transaction ([#1337](https://github.com/hirosystems/stacks-blockchain-api/issues/1337)) ([1edb256](1edb25697d))
* tests ([1c1fd16](1c1fd1619c))
* update testnet send-many-memo contract id ENV ([#1420](https://github.com/hirosystems/stacks-blockchain-api/issues/1420)) ([45ea24d](45ea24d9a2))
* upgrade stacks node versions to 2.05.0.3.0 ([#1328](https://github.com/hirosystems/stacks-blockchain-api/issues/1328)) ([e30636e](e30636e30f))
* use new `this.sqlTransaction(...)` in pox2 db queries ([27102da](27102da4d4))

### Reverts

* Revert "chore!: remove deprecated `/nft_events` endpoint (#1329)" (#1343) ([c537ee4](c537ee4c6f)), closes [#1329](https://github.com/hirosystems/stacks-blockchain-api/issues/1329) [#1343](https://github.com/hirosystems/stacks-blockchain-api/issues/1343)

### Miscellaneous Chores

* remove deprecated `/nft_events` endpoint ([#1329](https://github.com/hirosystems/stacks-blockchain-api/issues/1329)) ([65bb4e5](65bb4e55fa))

* chore: bump stacks-native-encoding-js version (#1495)

* chore: bump stacks-native-encoding-js version

* chore: revert auto-formatted line changes

* chore: bump stacks-native-encoding-js version

* Merge branch 'develop' into stacks-2.1

# Conflicts:
#	src/event-stream/event-server.ts

Co-authored-by: Rafael Cárdenas <rafael@rafaelcr.com>
Co-authored-by: CharlieC3 <2747302+CharlieC3@users.noreply.github.com>
Co-authored-by: semantic-release-bot <semantic-release-bot@martynus.net>
Co-authored-by: janniks <6362150+janniks@users.noreply.github.com>
Co-authored-by: janniks <janniks@users.noreply.github.com>
2022-12-23 15:49:57 +01:00
Rafael Cárdenas
c1dd9f14a4 chore: upgrade postgres.js to v3.3.1 (#1361) 2022-10-18 11:43:14 -05:00
Rafael Cárdenas
4ddb8f1aa0 chore: merge master 2022-09-07 14:06:30 -05:00
CharlieC3
2c8632fe8a ci: analyze commits using conventional commits 2022-08-26 12:14:49 -04:00
Matthew Little
aceb7afdc1 chore: npm audit dep updates (#1262)
* chore: npm audit dep updates

* chore: npm audit dep updates (client library)
2022-08-11 17:34:22 +02:00
Rafael Cárdenas
ff25ae057c chore: merge master 2022-08-08 12:27:25 -05:00
Matthew Little
d49f007b6e fix: arm64-alpine image, bump stacks-encoding-native-js #1217 (#1220) 2022-06-23 17:30:55 +02:00
Rafael Cárdenas
6e1eceb843 fix: mobx build dependencies (#1215)
* fix: cherry-pick mobx

* fix: swagger-cli dependency

* fix: take docs/package-lock.json from develop
2022-06-21 10:43:51 -05:00
Matthew Little
009fdb500b chore: merge develop 2022-06-16 14:54:17 +02:00
Rafael Cárdenas
5f8bc9fd4f fix: mobx breakage by locking package dependencies (#1206) 2022-06-16 14:41:23 +02:00
Rafael Cárdenas
1ae8fc5954 chore: merge master 2022-06-15 16:41:09 -05:00
Matthew Little
9b77ca22fc fix: contract-call txs returning null args in some situations, closes #1188 (#1192)
see https://github.com/hirosystems/stacks-encoding-native-js/pull/3 for more info
2022-06-01 16:10:06 +02:00
Matthew Little
f4d4733ad4 fix: use 128 max size when deserializing contract principals #1181 (#1182) 2022-05-25 19:00:56 +02:00
Rafael Cárdenas
3ff41779f8 feat: refactor pg classes, switch to postgres.js (#1148)
* refactor: split postgres-store into several files

* refactor: inherit all from PgStore

* refactor: start using PgStore everywhere

* refactor: fix test imports

* refactor: restore notifier into PgStore

* refactor: rename to PgWriteStore

* fix: tests and lint

* fix: unused exports

* refactor: pg-store converted

* refactor: pg-write-store

* refactor: use legacy pg for event requests, notifier and migrations

* refactor: update tests

* refactor: tx insert mode

* refactor: first test passing

* refactor: transformed some inserts

* refactor: transform all inserts

* refactor: fix token-tests

* refactor: some more tests fixed

* fix: more tests

* fix: nasty mempool query

* fix: microblock hash test

* fix: events weird test

* fix: api tests all passing

* fix: mb tests

* fix: bns code

* fix: datastore connection tests

* fix: rosetta, bns, token tests

* fix: faucet write db

* fix: write store to api server for faucet

* style: cleanup

* fix: faucet launch

* fix: upgrade postgres

* fix: test launch

* fix: remove type transform

* refactor: go back to buffer fields

* fix: insert numeric as string so values can fit

* refactor: move all types to common.ts

* style: comments

* fix: rosetta cli tests

* fix: undo launch custom

* refactor: bytea type, api tests pass

* fix: pg uri test

* chore: define PgNumeric type

* style: sql object references

* style: clean up tx events query

* style: add explicit PgJsonb type

* feat: dont parse into buffer from pg

* fix: datastore tests

* fix: rosetta tests

* fix: bns tests

* fix: microblock tests

* chore: add nvmrc set to v16.15.0

* chore: update package-lock.json

* chore: restore vscode config

* chore: shorten poll connection name to avoid hitting pg limits

* fix: remove returning statements when not required, use count instead

* fix: nvmrc back to v16

* fix: pool max env config

* refactor: move `PgNotifier` to use postgres lib (#1168)

* refactor: use postgres instead of pg-listen

* chore: remove pg-listen dependency

* fix: update to latest postgres lib

* fix: pool max env config

* fix: re-throw connection errors
2022-05-13 11:28:45 -05:00
Matthew Little
7747a9b5c7 chore(ci): upload code coverage to coveralls.io (#1146)
* chore(ci): upload code coverage to coveralls.io

* chore: enable coverage reporting on rosetta-cli tests
2022-04-15 08:54:31 +02:00
Matthew Little
f5c4da7b87 feat: [CPU optimizations] use native rust module for decoding Clarity values, binary transaction blobs, post-condition binary blobs, Stacks addresses (#1094)
* feat: use native nodejs addon module for Stacks binary encoding and decoding (Clarity values, binary wire formats, addresses, etc)

* refactor: move transaction, post-condition, and clarity value decoding from js to native addon

* chore: fix rosetta pox contract-call parsing

* chore: use npm registry for stacks-encoding-native-js

* chore: fix unused export

* chore: commit dev file to be converted to unit tests

* refactor: use stacks-encoding-native lib for stacks-address operations

* chore: bump stacks-encoding-native lib

* chore: stacks-native-encoding version bump with fix for decoding multisig tx sender address

* chore: stacks-native-encoding version bump with fix for decoding zero-length prefixed contract call args

* test: introduce tests for on-btc-chain originating stx tx processing

* chore: prettier ignore json payload sample files

* chore: clear up type import syntax

* test: tests for on-btc-chain originating stx-lock tx processing

* feat: expand cpu profiling capabilties with distinct start and stop endpoints easier to use in scripts

* chore: fix compilation error in /utils

* feat: support for performing cpu profiling with the native addon

* fix: avoid re-injecting sourcemap support when running with ts-node

* chore: updates for latest stacks-native-encoding library function usage, only decode clarity value type signatures when needed

* chore: decode only clarity value repr string rather than deep value in several areas

* chore: fix package-lock.json

* feat: commit load testing script with sampled endpoint list

* chore: fix synthetic-stx-txs-tests

* chore: update to latest stacks-encoding-native types (using hex strings rather than js buffers)

* chore: bump stacks-encoding-native version

* chore: fix regular integration tests

* chore: fix bug with rosetta principal repr parsing

* chore: bump stacks-encoding-native version

* chore: add several TODO(perf) comments where buffers should be hex strings

* chore: remove stacks-native-encoding debug code that have since been converted into unit tests

* chore: remove debug code

* fix: bug fix for reverse file stream bug if read chunk starts with line-ending

* chore: wrap synthetic stx txs tests
2022-04-13 18:51:33 +02:00
M Hassan Tariq
303eaaa8cd feat: pin exact dependencies in package.json (#1068)
* refactor: used exact dependencies

* refactor: updated dependencies to use the latest versions
2022-04-11 09:55:26 -05:00
M Hassan Tariq
cacf9a6fcc Remove memory store and increase test coverage (#1011)
* test: added block and status tests

* test: added rosetta tests

* refactor: removed memory-store.ts

* fix: test failing to exit

* fix: failing error in api-tests.ts

* fix: fixed api-tests.ts

* refactor: removed unecessory comments

* refactor: removed unecessary global server

* revert: unnecessary changes
2022-03-31 11:27:37 -06:00
Asim Mehmood
dba8e9a6ee chore: generate git info file with js script #910 (#986) 2022-02-03 14:51:40 +01:00
dependabot[bot]
685b4f9aab chore(deps): bump node-pg-migrate from 5.10.0 to 6.2.1 (#1018)
Bumps [node-pg-migrate](https://github.com/salsita/node-pg-migrate) from 5.10.0 to 6.2.1.
- [Release notes](https://github.com/salsita/node-pg-migrate/releases)
- [Changelog](https://github.com/salsita/node-pg-migrate/blob/master/CHANGELOG.md)
- [Commits](https://github.com/salsita/node-pg-migrate/compare/v5.10.0...v6.2.1)

---
updated-dependencies:
- dependency-name: node-pg-migrate
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-02-03 13:10:46 +01:00
Matthew Little
7ea3ae368f fix: npm audit fixes, dependency cleanup (#945) 2022-01-04 17:57:49 +01:00
CharlieC3
55bd5d44e5 ci: test installing client and docs deps 2021-12-22 13:22:15 -05:00
Matthew Little
0ddd18f248 chore: remove nodejs polyfill lib for abort-controller (#926) 2021-12-20 17:59:06 +01:00
Matthew Little
dc3588ec2d chore: upgrade to nodejs v16 #845 (#918) 2021-12-20 11:58:52 +01:00
Matthew Little
319e331954 refactor: remove @await/express usage #878 (#880) 2021-12-15 19:27:49 +01:00
Matthew Little
5697f6f477 chore: fix jest not quickly finishing tests (#888) 2021-12-09 22:01:16 +01:00
M Hassan Tariq
fdbc548e28 chore: lint and remove unused exports #806 2021-12-07 11:43:11 +01:00
Rafael Cárdenas
216b9ab715 feat: docker cleanup
* cleaning up docker

feat/cleanup needed it's own cleanup since it was so out of date. better to just start over with an up to date pull of develop

* Update paths for docker-compose files

* Updating paths and compose version

* switch network to testnet

* adjusting stacks-blockchain configs

* use correct context to build api image

* use the defaults from develop branch

* feat: update to latest stacks blockchain

* fix: rosetta output paths

* chore: update stacks api to v0.71.2

Co-authored-by: Charlie <2747302+CharlieC3@users.noreply.github.com>

* fix: remove magic_byte from mainnet toml

Co-authored-by: Charlie <2747302+CharlieC3@users.noreply.github.com>

* fix: update stacks node everywhere

Co-authored-by: wileyj <2847772+wileyj@users.noreply.github.com>
Co-authored-by: Charlie <2747302+CharlieC3@users.noreply.github.com>
2021-12-01 17:03:21 -06:00
Asim Mehmood
f764054cb8 feat: rosetta data api and construction validation with rosetta-cli
* test: added test for rosetta cli check:construction and check:data

* ci: added github workflow to test  roseta-cli:data

* ci: add github action for rosetta-cli: construction

* refactor: refactor cli tests

* fix: use localhoust for github actions in rosetta config file

* fix: github actions

* fix: rebase issue

* refactor: update rosetta-cli env files path

* refactor: remove hard coded cli container and use rosetta-cli ouput

* perf: remove unnecessary wait of 20 seconds
2021-11-16 10:18:04 -06:00
Matthew Little
285632a983 feat: stx addr encoding LRU cache 2021-11-16 12:39:25 +01:00