53 Commits

Author SHA1 Message Date
Matthew Little
285806f46c feat: pox-4 support (#1754)
* feat: handle pox-3 force unlocks

* chore: todo note

* feat: handle pox-3 force unlocks, pox-4 events, and revoke-delegate-stx event

* chore: rename pox-2 file

* chore: simplified pox route paths

* feat: add migration to create pox4_events table

* test: follow redirects in test fetch helper

* chore: fix storing different pox version events

* test: fix tests with missing pox4_events

* chore: move delegations lookup endpoint into pox router

* fix: handle pox4_events during reorgs

* test: fix pox4events reorg count

* chore: update stacks-node image to stacks 3.0

* test: begin transitioning pox tests to use pox-4

* chore: bump stacks-node to wip nakamoto branch

* test: switch more tests from pox-3 to pox-4

* fix: tx fee fall back in faucet

* test: update delegation check endpoints

* chore: isolate error in delegate-revoke test

* chore: pox3 to pox4 misc renames

* ci: rename 2.4 to 2.5

* test: remove no-longer applicable delegate-stx while stacking test

* chore: remove incorrect commend in pox4 test
2023-12-13 12:58:37 +01:00
Matthew Little
7c45f53622 feat: ingestion for TenureChange and NakamotoCoinbase tx types (#1753)
* feat: ingestion for TenureChange and NakamotoCoinbase tx types

* fix: add new tx fields to batch tx insert query

* chore: bump stacks-encoding-native-js

* test: fix flaky socket-io timeout test
2023-12-08 12:39:52 +01:00
Rafael Cárdenas
a490d673f8 refactor: remove old token metadata processor (#1747)
* refactor: remove old processor, start fetching data from contract

* refactor: remove old tests

* fix: tests and lint

* fix: use lru cache

* chore: delete old tables

* chore: fix comments [skip ci]

* fix: env comment block [skip ci]

* chore: avoid duplicate RPC lookups

* fix: make an incremental migration to drop tables

* fix: re-add metadata processing ENV flag

---------

Co-authored-by: Matthew Little <zone117x@gmail.com>
2023-11-09 10:11:26 -06:00
Matthew Little
3dec5d1b62 Merge branch 'master' into develop
# Conflicts:
#	.env
#	.github/workflows/ci.yml
#	.vscode/launch.json
#	package.json
#	src/api/init.ts
#	src/datastore/common.ts
#	src/datastore/pg-store.ts
#	src/event-stream/event-server.ts
#	src/event-stream/reader.ts
#	src/index.ts
#	src/test-utils/test-helpers.ts
#	src/tests-2.4/pox-3-delegate-aggregation.ts
#	src/tests-bns-e2e/bns-integration-tests.ts
#	src/tests-bns/v1-import-tests.ts
#	src/tests-tokens-metadata/tokens-metadata-tests.ts
2023-07-06 13:51:39 +02:00
Matthew Little
5d15bc8a8b Merge branch 'master' into develop
# Conflicts:
#	.vscode/tasks.json
2023-03-29 17:17:48 +02:00
Matthew Little
299705f270 fix: postgres should not be required in STACKS_API_MODE=offline mode #1391 (#1599) 2023-03-28 00:37:47 +02:00
Matthew Little
a724b384a6 Merge branch 'master' into develop 2023-03-22 20:23:56 +01:00
Rafael Cárdenas
6750861e99 fix: support genesis block 0 during BNS sync (#1559)
* fix: support genesis block 0 during BNS sync

* chore: remove unused export

* build: jest config path in event replay suite [skip ci]
2023-02-22 10:46:31 -06:00
Matthew Little
5d7056c1ba feat: support for subnets (#1549)
* test: initial integration test for subnets

* test: ensure first subnet block is mined

* feat: synthetic nft deposit tx

* test: increase timeout for subnets test

* test: attempt fix test flakiness in CI

* test: much faster subnet block mining time, subnet tests 5-10x faster

* ci: bump subnet-node docker image

* test: complete withdrawal on stacks chain

* test: try waiting for subnet contract interface to be ready

* test: verify NFT ownership on L1 after withdrawal

* test: fix subnet node port env vars

* feat: implement parsing for FT deposit synthetic tx

* chore: rename test file

* chore: do not skip test

* test: progress on subnet STX use-case test

* feat: implement parsing for STX deposit synthetic tx

* test: validation of synthetic tx for L1 deposit-stx

* test: validation of synthetic tx for L1 deposit-nft-asset

* test: validation of synthetic tx for L1 deposit-ft-asset

* chore: lint fix

* chore: remove debug comments
2023-02-14 13:30:41 +01:00
Matthew Little
dcbdfb9069 feat: Stacks 2.1 support (#1498)
* feat: support for Stacks 2.1 `versioned-smart-contract` transactions (#1279)

* dev: use latest `next` branch (Stacks 2.1) in docker-compose / dev environment

* chore: setup for testing with a local stacks-node v2.1 binary

* feat: support for versioned-smart-contract txs (Stacks 2.1)

* chore: update `stacks-encoding-native-js` dep to latest alpha

* fix: smart_contracts table missing the `clarity_version` column

* chore: test fixes

* chore: update test & dev env, and test suites to Stacks 2.1 image (#1280)

* fix: update several test suites to work with Stacks 2.1 docker image

* chore: move rosetta network endpoint tests to after more time-sensitive construction tests

* fix: attempt fix for `tests-rosetta-cli-data`

* chore: package.json fix

* fix: attempt fix for `tests-rosetta-cli-construction`

* chore: move test config into directory

* feat: separate suite for RPC request tests

* chore: update .github/workflows/ci.yml

Co-authored-by: Rafael Cárdenas <rafael@rafaelcr.com>

* chore: move btc faucet tests into own suite

* ci: add btc-faucet test to CI

Co-authored-by: Rafael Cárdenas <rafael@rafaelcr.com>

* chore: remove debug console.log

* chore: tests for versioned-smart-contact txs

* chore: update mocknet config to bootstrap to epoch 2.1

* chore: fix docker-compose background task logging in vscode debugger

* chore: remove local stacks-node debug and testing changes

Co-authored-by: Rafael Cárdenas <rafael@rafaelcr.com>

* feat: support for Stacks 2.1 `coinbase-pay-to-alt-recipient` txs (#1283)

* feat: support Stacks 2.1 `coinbase-pay-to-alt` tx types

* fix: add a `coinbase-to-alt` tx handling in a few more areas

* chore: fix coinbase in mempool table schema

* chore: update unit tests with new coinbase payload responses

* test: add tests for ingesting new coinbase tx payloads and generating expected API responses

* docs: better description for coinbase alt recipient

* feat: support for Stacks 2.1 stx-transfer event memo field (#1285)

* feat: support for Stacks 2.1 stx-transfer event memo field (the `stx-transfer-memo` Clarity2 function)

* fix: remove null memo field from different asset transfer events

* test: unit tests for Stacks 2.1 stx-transfer event memo field

* feat: include `stx-transfer-memo` results in `/address/<recipient>/stx_inbound` endpoint (#1289)

* feat: include Stacks 2.1 `stx-transfer-memo` in Rosetta token transfer operations (#1290)

* chore: cleanup Rosetta operation type strings

* feat: include `memo` from stx-transfer-memo events in Rosetta token transfer operations

* chore: lint fix

* chore: Rosetta tests for memo in stx-transfer-memo events

* fix: Rosetta memo strings should be decoded from hex

* chore: add `coinbase-pay-to-alt` tx type test for Rosetta parsing

* chore: upgrade to stacks.js v4.x (#1295)

* chore: upgrade to stacks.js v4.x

* chore: hardcode tx fees in rosetta tests

* chore: hardcode tx fees in token tests

* chore: hardcode tx fees in rosetta-cli-data tests

* chore: remove bugged fees in token tests

* chore: remove bugged fees in rosetta-cli-data tests

* chore: hardcode tx fees in rosetta-cli-data tests

* chore: fixes for bns tests

* feat: Stacks 2.1 PoX-2 support in Rosetta (#1339)

* feat: working setup for local krypton stacks-node miner with regtest bitcoind and auto-mining script

* feat: detect when stx mining starts for faster krypton stacks-node startup

* feat: use stacsk2.1-transition image for krypton debug deployment

* feat: update `/extended/v1/debug/broadcast/stack` endpoint to work with PoX-2

* chore: use PoX address decoding code from the WIP @stacks/stacking lib

* chore: update Rosetta PoX address parsing function

* chore(rosetta): skip extra /v2/info request when constructing stack-stx operation in Stacks 2.1

* feat: option in debug endpoint to use Rosetta to construct and submit Stacking transactions

* chore: simplify internal rosetta calls

* fix: Rosetta bug where parsing Stacking operations fails when the contract ABI is missing

* feat: Rosetta support for parsing PoX-2 contract calls

* feat: Rosetta support for parsing new Bitcoin address formats

* chore: remove no longer used testing configs and scripts

* chore: remove no longer used testing configs and scripts

* chore: debug page cleanup

* chore: configs for launching Stacks2.1-transition environment, and launching directly into Stacks2.1

* chore: task naming nit

* test: initial Stacks2.1 test suite

* chore: vscode debugger for 2.1 test suite

* ci: run 2.1 tests

* ci: possible fix for mapping host.docker.internal on linux

* chore: bitcoin address decoding without depending on try/catches for version detection

* chore: pin stacks2.1 images to hash

* chore: comment fix

* fix: report locked STX balance correctly when multiple event entries exist from stack-increase and stack-extend operations (#1363)

* chore: lint fixes

* chore: test moving migration files to non-ts source

* chore: rename migration file exts to js

* chore: fix pg migrations failing from certain jest scripts

* chore: cleanup jest.config.2.1.js

* chore: lint fix (revert)

* test: ensure locked stx balance event data has expected values after pox-2 stack-increase operation

* test: ensure stx unlock height event data has expected value after pox-2 stack-extend operation

* fix: report locked STX balance correctly when multiple event entries exist from stack-increase and stack-extend operations

* chore: refactor PoX-2 operation steps into separate tests

* chore: test scaffolding for pox-2 rewards

* chore: lint fixes

* test: validate Stacking slots and rewards after PoX-2 operations

* chore: update to bitcoin-v0.21.1 in stacks-2.1 test suite

* test: add additional 1 block padding to unlock height waiter for test flakiness

* chore(stacks-2.1): upgrade to bitcoinjs-lib-v6.x, initial P2TR work (#1367)

* chore: upgrade to bitcoinjs-lib-v6.x which has P2TR support

* feat: P2TR bitcoin address util

* chore: PoX-2 integration testing for segwit addresses (#1372)

* feat: functions for generating Bitcoin addresses in all formats supported by Stacks 2.1

* test: segwit PoX-2 integration tests

* test: e2e tests for PoX-2 segwit (P2WPKH) support

* chore: test PoX rewards against bitcoin json-rpc results

* test: e2e tests for PoX-2 rewards address formats p2pkh, p2sh, p2wpkh, p2wsh

* test: use more robust bitcoin `listtransactions` RPC method for validating PoX-2 rewards

* chore: update plain P2SH address generation to create a more standard P2SH(P2PKH) rather than 1-of-1 multisig address

* chore: temp skip segwit tests until `feat/native-segwit` stacks-blockchain branch is merged

* chore: Revert "temp skip segwit tests until `feat/native-segwit` stacks-blockchain branch is merged" -- too many other dependent changes

* chore: filter bitcoin rpc `listtransactions` result for only the address being tested

* ci: don't persist config in checkout

* feat: store Stacks block miner address value from new rewards payload introduced in https://github.com/stacks-network/stacks-blockchain/pull/3373 (#1413)

* chore: bump to latest Stacks 2.1 `next` branch (#1449)

* ci: configure semrel branches for 2.1 prerelease packages

* chore(release): 4.2.0-stacks-2.1.1 [skip ci]

# [4.2.0-stacks-2.1.1](https://github.com/hirosystems/stacks-blockchain-api/compare/v4.1.1...v4.2.0-stacks-2.1.1) (2022-11-22)

### Bug Fixes

* included query params in redirecting to prefix 0x in tx endpoint ([#1205](https://github.com/hirosystems/stacks-blockchain-api/issues/1205)) ([664cce7](664cce744d))
* incorrect websocket/socket.io transaction updates ([#1197](https://github.com/hirosystems/stacks-blockchain-api/issues/1197)) ([8ee1da8](8ee1da840b))
* mobx breakage by locking package dependencies ([#1206](https://github.com/hirosystems/stacks-blockchain-api/issues/1206)) ([5f8bc9f](5f8bc9fd4f))
* optimize `getMicroblocks` query ([#1179](https://github.com/hirosystems/stacks-blockchain-api/issues/1179)) ([7691109](769110926e))
* optimize block endpoint ([#1190](https://github.com/hirosystems/stacks-blockchain-api/issues/1190)) ([943e2d1](943e2d1c55))
* remove duplicate txs in microblock responses ([#1167](https://github.com/hirosystems/stacks-blockchain-api/issues/1167)) ([15c0c11](15c0c1124a))
* report locked STX balance correctly when multiple event entries exist from stack-increase and stack-extend operations ([#1363](https://github.com/hirosystems/stacks-blockchain-api/issues/1363)) ([d06beb3](d06beb32f1))
* upsert nft and ft metadata ([#1193](https://github.com/hirosystems/stacks-blockchain-api/issues/1193)) ([c4eec5d](c4eec5d060))

### Features

* add `transaction_count` for `microblocks_accepted` in block ([#1162](https://github.com/hirosystems/stacks-blockchain-api/issues/1162)) ([78d7d9c](78d7d9c9f7))
* add API version in response header ([#1216](https://github.com/hirosystems/stacks-blockchain-api/issues/1216)) ([1e998db](1e998db7d1))
* include `stx-transfer-memo` results in `/address/<recipient>/stx_inbound` endpoint ([#1289](https://github.com/hirosystems/stacks-blockchain-api/issues/1289)) ([cb5ca93](cb5ca933fe))
* include Stacks 2.1 `stx-transfer-memo` in Rosetta token transfer operations ([#1290](https://github.com/hirosystems/stacks-blockchain-api/issues/1290)) ([c7cc900](c7cc900edf))
* mempool stats endpoint and prometheus metrics ([#1241](https://github.com/hirosystems/stacks-blockchain-api/issues/1241)) ([9482238](9482238599))
* refactor pg classes, switch to postgres.js ([#1148](https://github.com/hirosystems/stacks-blockchain-api/issues/1148)) ([3ff4177](3ff41779f8)), closes [#1168](https://github.com/hirosystems/stacks-blockchain-api/issues/1168)
* send nft updates through websocket channels ([#1218](https://github.com/hirosystems/stacks-blockchain-api/issues/1218)) ([920a7b8](920a7b892a))
* Stacks 2.1 PoX-2 support in Rosetta ([#1339](https://github.com/hirosystems/stacks-blockchain-api/issues/1339)) ([e04bf64](e04bf64e6f))
* store Stacks block miner address value from new rewards payload introduced in https://github.com/stacks-network/stacks-blockchain/pull/3373 ([#1413](https://github.com/hirosystems/stacks-blockchain-api/issues/1413)) ([e34fab5](e34fab5e5d))
* support for Stacks 2.1 `coinbase-pay-to-alt-recipient` txs ([#1283](https://github.com/hirosystems/stacks-blockchain-api/issues/1283)) ([7e5752a](7e5752a7e6))
* support for Stacks 2.1 `versioned-smart-contract` transactions ([#1279](https://github.com/hirosystems/stacks-blockchain-api/issues/1279)) ([4ff1a91](4ff1a913ed)), closes [#1280](https://github.com/hirosystems/stacks-blockchain-api/issues/1280)
* support for Stacks 2.1 stx-transfer event memo field ([#1285](https://github.com/hirosystems/stacks-blockchain-api/issues/1285)) ([67332ce](67332ce699))

* feat: ingestion and querying for new PoX-2 events (#1403)

* test: Rosetta PoX-2 rewards and unlock events

* feat: progress on parsing and writing PoX2 events to postgres

* feat: sql insert for pox2 event data

* feat: reading pox2 event data from db, sql constraints on named pox event types

* feat: db-controller and initial GET route for reading & parsing pox2 events

* chore: progress on e2e testing for pox-2 delegation functionality

* chore: e2e testing for pox-2 delegation commit operation

* chore: e2e testing for pox-2 delegate-stack-increase

* chore: e2e testing for pox-2 delegate-stack-extend

* chore: e2e testing for pox-2 handle-unlock (missed reward slot)

* chore: lint fix, possible test fix

* chore: validation tests for each of the pox2-event payloads

* chore: potential test fixes

* chore: test fixes num 2

* chore: additional flaky test fixes

* feat: use pox2-events table to calculate account STX locked amount in /extended/v1/address balance endpoints

* chore: Rosetta integration tests for using segwit address in stacking operation

* test: ensure stacks RPC reports expected locked balance after Rosetta lock op

* chore: bump to latest pox2-events stacks-node image

* chore: move pox2-event parsing code into separate file

* ci: bump stacks2.1 test suite time to account for longer block mining interval from latest stacks-node changes

* chore: speed up tests with better "wait for pox cycle" logic

* ci: test fixes

* fix: adjustments to `handle-unlock` processing

* feat: add support for latest additions to pox2: `stack-aggregation-commit-indexed` and `stack-aggregation-increase`

* chore: integration tests for `stack-aggregation-commit-indexed` and `stack-aggregation-increase` pox2 event handling

* feat: store pox_v1_unlock_height as singleton in pg

* chore: initial setup for stacks-2.1 transition test suite

* chore: fix pox_state migrations

* feat: make account balance drop pox1-locked state after pox_v1_unlock_height is reached

* fix: rosetta block unlock events were limited to one account

* fix: off-by-one bug causing accounts to be reported as unlocked one block too early

* feat: add burn_block_height to db chainTip and /extended/status

* chore: fix datastore-tests from new status endpoint field

* chore: fix datastore-tests from new status endpoint field, second pass

* fix: adjust pox2_events query to handle special case with `handle-unlock`

* feat: add `contract_name` processing for the `stx_lock_event`, see https://github.com/stacks-network/stacks-blockchain/issues/3413

* chore: bump to latest https://github.com/stacks-network/stacks-blockchain/pull/3415

* chore: add account balance test for consistency between RPC and API balances throughout pox transition

* ci: matrix for stacks2.1-transition tests

* chore: start of Rosetta pox transition period tests

* chore: test Rosetta account balance endpoint in pox-2 stacking test suite

* ci: use unique account key for pox-2 rosetta test suite

* chore: test for pox_v1_unlock_height in Rosetta pox transition tests

* chore: refactor giant pox-2 test file into multiple files, run separately in ci

* feat: generate synthetic Rosetta stx unlock operations for the pox_v1_unlock_height block

* feat: generate Rosetta unlock operations from pox2 auto-unlock events

* fix: store locked balance from `handle-unlock` event rather than resetting to zero

* chore: remove todos

* chore: remove todos

* chore: remove no-longer correct sanity check

* chore: address PR feedback, composite index for pox2_events ordering

* fix: add default stx faucet tx fee if estimate not available (#1456)

* test: add integration test for stx faucet requests

* fix: add default stx faucet tx fee if estimate not available

* ci: disable generated doc type file test for now

* fix: add automatic pox switching to rosetta (#1454)

* refactor: add minor changes to rosetta

* chore: switch to stacks.js v6 and refactor helpers

* fix: update rosetta construction for 2.1

* test: refactor pox transition tests

* test: add btc address format rosetta test

* build: npm audit

* chore: lint fix json files not in src

* chore: fix exports

* test: add rosetta stacking for each block height

* test: add double stacking test

* chore: revert back to original suggested_fee_multiplier behavior

* chore: remove dynamic contract from dummy data

* chore: remove todos

* test: fix test address

* test: skip failing tests

* test: add check to rosetta contract tests

* chore: update to latest node next branch for testing

Co-authored-by: janniks <janniks@users.noreply.github.com>

* chore(release): 7.0.0-stacks-2.1.1 [skip ci]

# [7.0.0-stacks-2.1.1](https://github.com/hirosystems/stacks-blockchain-api/compare/v7.0.0-beta.1...v7.0.0-stacks-2.1.1) (2022-11-29)

### Bug Fixes

* add automatic pox switching to rosetta ([#1454](https://github.com/hirosystems/stacks-blockchain-api/issues/1454)) ([ad7e492](ad7e49216b))
* add default stx faucet tx fee if estimate not available ([#1456](https://github.com/hirosystems/stacks-blockchain-api/issues/1456)) ([eeeffd0](eeeffd0244))

### Features

* ingestion and querying for new PoX-2 events ([#1403](https://github.com/hirosystems/stacks-blockchain-api/issues/1403)) ([1936ba6](1936ba6519))

* ci: re-enable docs-lint

* chore: fix post-merge lint errors

* fix: use new `this.sqlTransaction(...)` in pox2 db queries

* ci: re-enable docs-lint

* chore: post-merge fixes

* ci: add missing gh job for `event-replay` test suite

* ci: increase timeout for 2.1 test suite

* chore: split up pox-2-rosetta tests (#1461)

* chore: split up pox-2-rosetta tests

* chore: split up pox-2-rosetta test suites into smaller units

* feat: [Stacks 2.1] Support new "block 0" boot events (#1476)

* chore: split up pox-2-rosetta tests

* chore: split up pox-2-rosetta test suites into smaller units

* feat: new regtest standalone image with 2.1 node

* chore: store postgres data in same volume

* chore: add configurable heights for each epoch

* feat: arg to toggle if docker image is bootstraps the chainstate

* chore: block height adjustments

* chore: block heights that work with epoch 2.0 and 2.05 receipts

* chore: fix bash script error

* feat: store and forward boot events

* chore: allow pg server connections

* chore: do not error when skipping bootstrap

* chore: update to latest stacks2.1 images

* chore: add epoch config for mocknet node

* ci: attempt fix for host.docker.internal for tests using stacks mocknet image

* ci: fix tokens and rosetta mocknet mode tests

* feat: finish ingesting "block 0" data once the first real block is received

* test: implement test suite to ensure "block-zero" is ingested correctly

* test: re-enable double-stacking test suite

* ci: add workflow dispatch for standalone docker image build [skip ci]

* ci: allow use of existing stacks-node binaries from docker context

* ci: allow stacks-api version tag to be set, require git sha args

* ci: add workflow dispatch for standalone docker image build [skip ci]

* chore: bump to latest stacks-blockchain `next` branch

* chore(release): 7.0.0-stacks-2.1.2 [skip ci]

## [7.0.0-stacks-2.1.2](https://github.com/hirosystems/stacks-blockchain-api/compare/v7.0.0-stacks-2.1.1...v7.0.0-stacks-2.1.2) (2022-12-21)

### ⚠ BREAKING CHANGES

* remove deprecated `/nft_events` endpoint (#1329)
* mark breaking change
* optimize tables and improve canonical treatment of BNS data (#1287)

### Features

* [Stacks 2.1] Support new "block 0" boot events ([#1476](https://github.com/hirosystems/stacks-blockchain-api/issues/1476)) ([3d1b8f6](3d1b8f6f86))
* add ENV configs for DB close and API shutdown timeouts ([#1366](https://github.com/hirosystems/stacks-blockchain-api/issues/1366)) ([444f008](444f008fe2))
* add indexes for index_block_hash on BNS tables ([#1304](https://github.com/hirosystems/stacks-blockchain-api/issues/1304)) ([bbf4b2d](bbf4b2d2b8))
* add owner index to subdomains table ([#1331](https://github.com/hirosystems/stacks-blockchain-api/issues/1331)) ([a6c5e12](a6c5e12faa))
* add token_type metadata for rosetta ft operations ([#1332](https://github.com/hirosystems/stacks-blockchain-api/issues/1332)) ([09af27b](09af27b24b))
* **agg-paging-limits:** aggregated all paging query limits ([#1401](https://github.com/hirosystems/stacks-blockchain-api/issues/1401)) ([0203d36](0203d36342)), closes [#1379](https://github.com/hirosystems/stacks-blockchain-api/issues/1379) [#1379](https://github.com/hirosystems/stacks-blockchain-api/issues/1379)
* configurable pg connection lifetime and idle timeouts ([#1355](https://github.com/hirosystems/stacks-blockchain-api/issues/1355)) ([46ccf06](46ccf0640d))
* mark breaking change ([669fd0d](669fd0d8c0))
* optimize tables and improve canonical treatment of BNS data ([#1287](https://github.com/hirosystems/stacks-blockchain-api/issues/1287)) ([1f64818](1f648187b8))

### Bug Fixes

* add memos to send-many-memo rosetta STX transfer operations ([#1389](https://github.com/hirosystems/stacks-blockchain-api/issues/1389)) ([0a552b8](0a552b8d8c))
* add owner index on subdomains table ([#1323](https://github.com/hirosystems/stacks-blockchain-api/issues/1323)) ([c9c6d05](c9c6d053fd))
* add postgres connection error checking for ECONNRESET code ([03a1896](03a1896cff))
* bump version ([3863cce](3863cce1a6))
* catch cache controller db errors ([#1368](https://github.com/hirosystems/stacks-blockchain-api/issues/1368)) ([f15df41](f15df41fa9))
* catch pg exceptions on queries outside of express ([#1348](https://github.com/hirosystems/stacks-blockchain-api/issues/1348)) ([1f07b85](1f07b8587c))
* consolidate db migrations ([#1314](https://github.com/hirosystems/stacks-blockchain-api/issues/1314)) ([d6bdf9f](d6bdf9faff))
* detect name transfers and renewals in special circumstances ([#1303](https://github.com/hirosystems/stacks-blockchain-api/issues/1303)) ([cd381a9](cd381a95b4))
* disable faucet endpoints on mainnet ([#1425](https://github.com/hirosystems/stacks-blockchain-api/issues/1425)) ([b79b9b4](b79b9b43d5))
* event_observer_requests json writes ([#1334](https://github.com/hirosystems/stacks-blockchain-api/issues/1334)) ([465aa0b](465aa0b42c))
* filter BNS processing for successful txs only ([#1309](https://github.com/hirosystems/stacks-blockchain-api/issues/1309)) ([6a12936](6a129369c6))
* get rosetta latest block from chain_tip view ([#1445](https://github.com/hirosystems/stacks-blockchain-api/issues/1445)) ([ad386d3](ad386d30d1))
* guarantee db is empty before performing a replay ([#1374](https://github.com/hirosystems/stacks-blockchain-api/issues/1374)) ([ef8e7a9](ef8e7a9185))
* guard against empty lists before querying postgres ([#1345](https://github.com/hirosystems/stacks-blockchain-api/issues/1345)) ([6c88a16](6c88a166c8))
* handle pg exceptions on web socket transmitter ([#1353](https://github.com/hirosystems/stacks-blockchain-api/issues/1353)) ([2e6448d](2e6448d7af))
* handle postgres dns lookup error ([#1433](https://github.com/hirosystems/stacks-blockchain-api/issues/1433)) ([e00efd4](e00efd4e64))
* handle postgres.js connection timeouts ([#1424](https://github.com/hirosystems/stacks-blockchain-api/issues/1424)) ([4a2a342](4a2a342b56))
* handle websocket messages with a priority queue ([#1427](https://github.com/hirosystems/stacks-blockchain-api/issues/1427)) ([f0cb01c](f0cb01c054))
* import BNS v1 data during event replay ([#1301](https://github.com/hirosystems/stacks-blockchain-api/issues/1301)) ([bc59817](bc59817aa9))
* lint docs ci dependencies ([#1458](https://github.com/hirosystems/stacks-blockchain-api/issues/1458)) ([90d0c7b](90d0c7b831))
* log PoisonMicroblock tx instead rather than throwing ([#1379](https://github.com/hirosystems/stacks-blockchain-api/issues/1379)) ([cee6352](cee63529b4))
* refresh materialized views concurrently ([#1270](https://github.com/hirosystems/stacks-blockchain-api/issues/1270)) ([057c541](057c541b8c))
* refresh materialized views concurrently in new pg format ([#1324](https://github.com/hirosystems/stacks-blockchain-api/issues/1324)) ([20b284f](20b284fa38))
* refresh materialized views in their own pg connection ([#1356](https://github.com/hirosystems/stacks-blockchain-api/issues/1356)) ([9433d3c](9433d3c9c2))
* remove duplicate tx socket updates inside db transactions ([#1360](https://github.com/hirosystems/stacks-blockchain-api/issues/1360)) ([60c185d](60c185d839))
* remove live tsv append ([#1315](https://github.com/hirosystems/stacks-blockchain-api/issues/1315)) ([e2a1247](e2a124710f))
* retry pg connection on new library code ([#1326](https://github.com/hirosystems/stacks-blockchain-api/issues/1326)) ([35db939](35db939199))
* revert to 404 error code on bns name errors ([#1440](https://github.com/hirosystems/stacks-blockchain-api/issues/1440)) ([cdc039c](cdc039cea8))
* skip migrations on read-only start ([#1351](https://github.com/hirosystems/stacks-blockchain-api/issues/1351)) ([1d32261](1d322614e7))
* sql transaction consistency ([#1410](https://github.com/hirosystems/stacks-blockchain-api/issues/1410)) ([01e26d9](01e26d9c89))
* support multiple BNS name events in the same transaction ([#1337](https://github.com/hirosystems/stacks-blockchain-api/issues/1337)) ([1edb256](1edb25697d))
* tests ([1c1fd16](1c1fd1619c))
* update testnet send-many-memo contract id ENV ([#1420](https://github.com/hirosystems/stacks-blockchain-api/issues/1420)) ([45ea24d](45ea24d9a2))
* upgrade stacks node versions to 2.05.0.3.0 ([#1328](https://github.com/hirosystems/stacks-blockchain-api/issues/1328)) ([e30636e](e30636e30f))
* use new `this.sqlTransaction(...)` in pox2 db queries ([27102da](27102da4d4))

### Reverts

* Revert "chore!: remove deprecated `/nft_events` endpoint (#1329)" (#1343) ([c537ee4](c537ee4c6f)), closes [#1329](https://github.com/hirosystems/stacks-blockchain-api/issues/1329) [#1343](https://github.com/hirosystems/stacks-blockchain-api/issues/1343)

### Miscellaneous Chores

* remove deprecated `/nft_events` endpoint ([#1329](https://github.com/hirosystems/stacks-blockchain-api/issues/1329)) ([65bb4e5](65bb4e55fa))

* chore: bump stacks-native-encoding-js version (#1495)

* chore: bump stacks-native-encoding-js version

* chore: revert auto-formatted line changes

* chore: bump stacks-native-encoding-js version

* Merge branch 'develop' into stacks-2.1

# Conflicts:
#	src/event-stream/event-server.ts

Co-authored-by: Rafael Cárdenas <rafael@rafaelcr.com>
Co-authored-by: CharlieC3 <2747302+CharlieC3@users.noreply.github.com>
Co-authored-by: semantic-release-bot <semantic-release-bot@martynus.net>
Co-authored-by: janniks <6362150+janniks@users.noreply.github.com>
Co-authored-by: janniks <janniks@users.noreply.github.com>
2022-12-23 15:49:57 +01:00
Rafael Cárdenas
ef8e7a9185 fix: guarantee db is empty before performing a replay (#1374)
* feat: add event-replay test suite

* test: bns genesis block

* chore: rename tsv to mainnet

* test: export import cycle

* chore: reduce mainnet tsv size

* fix: db has data check

* fix: drop views first, tables second

* feat: friendly error when migration cycle failed
2022-10-27 15:29:56 -05:00
Rafael Cárdenas
1ae8fc5954 chore: merge master 2022-06-15 16:41:09 -05:00
Rafael Cárdenas
3a0ead18fd fix: propagate chain id correctly to bns router (#1180) 2022-05-20 10:15:00 -05:00
Rafael Cárdenas
3ff41779f8 feat: refactor pg classes, switch to postgres.js (#1148)
* refactor: split postgres-store into several files

* refactor: inherit all from PgStore

* refactor: start using PgStore everywhere

* refactor: fix test imports

* refactor: restore notifier into PgStore

* refactor: rename to PgWriteStore

* fix: tests and lint

* fix: unused exports

* refactor: pg-store converted

* refactor: pg-write-store

* refactor: use legacy pg for event requests, notifier and migrations

* refactor: update tests

* refactor: tx insert mode

* refactor: first test passing

* refactor: transformed some inserts

* refactor: transform all inserts

* refactor: fix token-tests

* refactor: some more tests fixed

* fix: more tests

* fix: nasty mempool query

* fix: microblock hash test

* fix: events weird test

* fix: api tests all passing

* fix: mb tests

* fix: bns code

* fix: datastore connection tests

* fix: rosetta, bns, token tests

* fix: faucet write db

* fix: write store to api server for faucet

* style: cleanup

* fix: faucet launch

* fix: upgrade postgres

* fix: test launch

* fix: remove type transform

* refactor: go back to buffer fields

* fix: insert numeric as string so values can fit

* refactor: move all types to common.ts

* style: comments

* fix: rosetta cli tests

* fix: undo launch custom

* refactor: bytea type, api tests pass

* fix: pg uri test

* chore: define PgNumeric type

* style: sql object references

* style: clean up tx events query

* style: add explicit PgJsonb type

* feat: dont parse into buffer from pg

* fix: datastore tests

* fix: rosetta tests

* fix: bns tests

* fix: microblock tests

* chore: add nvmrc set to v16.15.0

* chore: update package-lock.json

* chore: restore vscode config

* chore: shorten poll connection name to avoid hitting pg limits

* fix: remove returning statements when not required, use count instead

* fix: nvmrc back to v16

* fix: pool max env config

* refactor: move `PgNotifier` to use postgres lib (#1168)

* refactor: use postgres instead of pg-listen

* chore: remove pg-listen dependency

* fix: update to latest postgres lib

* fix: pool max env config

* fix: re-throw connection errors
2022-05-13 11:28:45 -05:00
Matthew Little
f5c4da7b87 feat: [CPU optimizations] use native rust module for decoding Clarity values, binary transaction blobs, post-condition binary blobs, Stacks addresses (#1094)
* feat: use native nodejs addon module for Stacks binary encoding and decoding (Clarity values, binary wire formats, addresses, etc)

* refactor: move transaction, post-condition, and clarity value decoding from js to native addon

* chore: fix rosetta pox contract-call parsing

* chore: use npm registry for stacks-encoding-native-js

* chore: fix unused export

* chore: commit dev file to be converted to unit tests

* refactor: use stacks-encoding-native lib for stacks-address operations

* chore: bump stacks-encoding-native lib

* chore: stacks-native-encoding version bump with fix for decoding multisig tx sender address

* chore: stacks-native-encoding version bump with fix for decoding zero-length prefixed contract call args

* test: introduce tests for on-btc-chain originating stx tx processing

* chore: prettier ignore json payload sample files

* chore: clear up type import syntax

* test: tests for on-btc-chain originating stx-lock tx processing

* feat: expand cpu profiling capabilties with distinct start and stop endpoints easier to use in scripts

* chore: fix compilation error in /utils

* feat: support for performing cpu profiling with the native addon

* fix: avoid re-injecting sourcemap support when running with ts-node

* chore: updates for latest stacks-native-encoding library function usage, only decode clarity value type signatures when needed

* chore: decode only clarity value repr string rather than deep value in several areas

* chore: fix package-lock.json

* feat: commit load testing script with sampled endpoint list

* chore: fix synthetic-stx-txs-tests

* chore: update to latest stacks-encoding-native types (using hex strings rather than js buffers)

* chore: bump stacks-encoding-native version

* chore: fix regular integration tests

* chore: fix bug with rosetta principal repr parsing

* chore: bump stacks-encoding-native version

* chore: add several TODO(perf) comments where buffers should be hex strings

* chore: remove stacks-native-encoding debug code that have since been converted into unit tests

* chore: remove debug code

* fix: bug fix for reverse file stream bug if read chunk starts with line-ending

* chore: wrap synthetic stx txs tests
2022-04-13 18:51:33 +02:00
M Hassan Tariq
cacf9a6fcc Remove memory store and increase test coverage (#1011)
* test: added block and status tests

* test: added rosetta tests

* refactor: removed memory-store.ts

* fix: test failing to exit

* fix: failing error in api-tests.ts

* fix: fixed api-tests.ts

* refactor: removed unecessory comments

* refactor: removed unecessary global server

* revert: unnecessary changes
2022-03-31 11:27:37 -06:00
Rafael Cárdenas
adf3821821 feat: add write-only mode 2022-01-13 14:27:33 -06:00
Matthew Little
9c10bf70f7 chore: util for testing chaintip-based http cache-control behavior against CDNs (#890) 2021-12-15 17:18:41 +01:00
Rafael Cárdenas
d1adca4d50 feat: add read-only mode
* feat: add readonly vscode launch config and .env var

* feat: add pg-listen dependency

* feat: first successful messages

* chore: modify event emitter types to use only ids

* chore: apply changes to ws rpc

* chore: reduce size of address tx updates

* fix: build tests

* chore: move pg-listen logic to separate class

* fix: incorrect message forwarding

* chore: remove special treatment for test env

* fix: rosetta, bns, tokens tests

* fix: ws-rpc tests and serialization

* fix: block height on ws rpc test

* fix: remove manual notifier connection

* fix: remove circular dependency on PgNotifier

* fix: move away from Map and Set

* fix: close pg channels correcty before disconnecting

* fix: update socket-io with latest messages

* chore: reduce size of token metadata messages

* chore: remove logs on readonly task

* chore: restore old style on rosetta tests

* chore: remove tx_id array from block update message

* feat: don't send tx events on address updates

* fix: remove logs from devenv docker

* chore: remove unused interface

* fix: skip db migrations during read-only mode

* chore: compound debug for dual reader-writer mode

* chore: restore removed line

* chore: delete unused docker config

* chore: update ts string management

* chore: remove readonly npm script

* chore: don't use notifier on event replay

* chore: make notifications be non-blocking

* feat: ignore limit/offset when getting asset transfers for a single block

* chore: make limit and offset optional args too

* chore: fix websocket unit test

* fix: fetch block txs correctly

Co-authored-by: Matthew Little <zone117x@gmail.com>
2021-09-29 12:11:49 -05:00
Asim Mehmood
33f11bbcf3 feat: token metadata 2021-09-01 18:14:05 -05:00
Matthew Little
912113ded8 feat: option to export all raw event observer requests to file via program args, progress on replaying requests through the observer interface 2021-07-15 13:07:39 -05:00
Matthew Little
6b644dd3cc chore: more accurate doc type generation 2021-07-15 13:07:39 -05:00
Matthew Little
9d5e740146 chore: stricter error handling and more explicit shutdown behavior 2021-05-11 12:51:29 +02:00
Matthew Little
6a91ad98c9 fix: bns v1-import test 2021-03-17 10:22:41 +01:00
Matthew Little
cee0547d01 fix: chainid env var to work around bootup deadlock 2021-01-26 16:36:11 +01:00
Matthew Little
334eb9da9d feat: integrate stx lock event handling into db 2020-11-12 18:20:29 +01:00
Matthew Little
f07bc57553 feat: add follower-mode vscode debug config 2020-09-25 19:24:10 +02:00
Faried Nawaz
93c1538209 test: add rosetta tests
The tests can be executed with

    npm run test:rosetta

Added by @zone117x in https://github.com/fariedt/stacks-blockchain-api/pull/2
with two minor changes:

1. uses the node version from "process.env" and the middleware version
from package.json for /rosetta/v1/network/options

2. adds a full block_identifier field (index + hash) to the
invalid transcation hash test.
2020-09-08 18:23:05 +02:00
Matthew Little
d12ba53fb0 feat: initial debug endpoint support for sending multisig transactions 2020-08-24 17:56:01 -06:00
Matthew Little
6eda93d7ff feat: auto-generated client demo 2020-08-20 16:20:44 -06:00
Matthew Little
67baf1e43b chore: remove sidecar usage from database env vars 2020-07-27 18:43:46 +02:00
Matthew Little
f751c7b294 chore: add error details for failed core-node proxy endpoints 2020-05-07 16:03:45 +02:00
Matthew Little
52e1f96c8a feat: reading events from postgres 2020-05-04 12:09:00 +02:00
Matthew Little
7227b7fc6b feat: pg schema and insert support for asset events, contract logs, and contract deploy payloads 2020-05-04 12:09:00 +02:00
Matthew Little
86be553fe6 fix: postcondition serialization issues + unit tests 2020-04-24 17:27:15 +02:00
Matthew Little
0c0e148072 fix: update to latest tx encoding 2020-04-23 15:06:12 +02:00
Matthew Little
c9c820bfb0 feat: running core-node in follower mode 2020-04-23 15:06:12 +02:00
Matthew Little
24b7997d17 fix: (2nd attempt) fix for 504 errors when reponse takes several seconds
Use a more maintained express proxy library
2020-04-22 12:19:18 +02:00
Matthew Little
90b8574cb4 feat: proxy core-node rpc endpoints 2020-04-17 17:15:15 +02:00
Matthew Little
07fc566db6 feat: postgres launched via docker-compose in dev env 2020-04-15 11:43:50 +02:00
Matthew Little
5e02f61764 feat: docker-compose dev & testing for core-node 2020-04-15 11:43:50 +02:00
Matthew Little
450136e09d fix: mempool path shared between docker services & faster Dockerfile build 2020-04-15 11:43:50 +02:00
Matthew Little
6bf3faeb8a feat: docker-compose working for sidecar + core-node + postgres 2020-04-15 11:43:50 +02:00
Matthew Little
ddd694cdbf feat: blockstack-core build and distribute util, docker-compose setup for postgres and core-node dependency services 2020-04-15 11:43:50 +02:00
kyranjamie
dd6be80782 feat: add schema validation to api responses 2020-04-10 09:15:49 +02:00
kyranjamie
b05d571a88 feat: render types from json schema 2020-04-06 11:39:59 +02:00
Matthew Little
e301c1aefc feat: db schema definitions and sql for tx entries 2020-03-26 15:50:30 +01:00
Matthew Little
974f668bba feat: initial postgres datastore setup 2020-03-19 12:08:08 +01:00
Matthew Little
0344cb5086 feat: handling for contract-call tx type data extraction 2020-03-18 13:10:53 +01:00
Matthew Little
d7752c9cc2 feat: parsing Clarity values in contract-call tx and nonfungible post-conditions using stacks-transactions-js 2020-03-16 21:38:56 +01:00