* chore: cargo fmt * fix: making db access more defensive * chore: clippy * fix: reopen conn on each iteration * fix: retry mechanism for statement prepare * refactor: sqlite requests retries * feat: set explicit timeout value * fix: make logging optional * chore: logger cleaning * fix: result looping * feat: introduce ORDHOOK_MAINTENANCE mode, revisit cli options * chore: re-qualify logs * chore: enable mode code site * chore: re-qualify log * fix: keep trying opening conn * fix: improve rocksdb resiliency with retries * fix: baseline experiment * chore: update chainhook-sdk * fix: testnet support * Squashed commit of the following: commita84c9517cfMerge:aaef4b21ec9374Author: Ludo Galabru <ludo@hiro.so> Date: Thu Nov 23 11:33:52 2023 -0500 Merge branch 'main' into develop commitaaef4b25dfAuthor: Ludo Galabru <ludo@hiro.so> Date: Thu Nov 23 11:32:47 2023 -0500 chore: update dependencies commit1ec9374c3aAuthor: semantic-release-bot <semantic-release-bot@martynus.net> Date: Thu Nov 23 12:33:35 2023 +0000 chore(release): 1.1.0 [skip ci] ## [1.1.0](https://github.com/hirosystems/ordhook/compare/v1.0.1...v1.1.0) (2023-11-23) ### Features * ordhook-sdk-js refactoring ([#186](https://github.com/hirosystems/ordhook/issues/186)) ([0d145df](0d145dfb89)) ### Bug Fixes * add destination to transfer events - release v1.0.2 ([47f365e](47f365eb47)), closes [#170](https://github.com/hirosystems/ordhook/issues/170) [#171](https://github.com/hirosystems/ordhook/issues/171) [#285](https://github.com/hirosystems/ordhook/issues/285) [#310](https://github.com/hirosystems/ordhook/issues/310) [#168](https://github.com/hirosystems/ordhook/issues/168) [#173](https://github.com/hirosystems/ordhook/issues/173) [#175](https://github.com/hirosystems/ordhook/issues/175) [#178](https://github.com/hirosystems/ordhook/issues/178) [#182](https://github.com/hirosystems/ordhook/issues/182) [#183](https://github.com/hirosystems/ordhook/issues/183) * build error / warning ([055c0d7](055c0d78d6)) * ci ([ac3d458](ac3d4580f9)) * CI rust version mismatch, create empty db ([#173](https://github.com/hirosystems/ordhook/issues/173)) ([cd2842e](cd2842eac7)) * databases lock ([d0b57c5](d0b57c5771)) * enable streaming for in-memory observers ([#171](https://github.com/hirosystems/ordhook/issues/171)) ([50f8393](50f8393ae3)) * grammar tweaks ([54e5fa1](54e5fa1321)) * grammar tweaks ([e50aef0](e50aef00b4)) * grammar updates ([66a4559](66a4559aec)) * initial flow ([#178](https://github.com/hirosystems/ordhook/issues/178)) ([8bb24be](8bb24beb9a)) * release 1.0.2 ([#179](https://github.com/hirosystems/ordhook/issues/179)) ([ec1f28e](ec1f28ea50)), closes [#170](https://github.com/hirosystems/ordhook/issues/170) [#171](https://github.com/hirosystems/ordhook/issues/171) [#285](https://github.com/hirosystems/ordhook/issues/285) [#310](https://github.com/hirosystems/ordhook/issues/310) [#168](https://github.com/hirosystems/ordhook/issues/168) [#173](https://github.com/hirosystems/ordhook/issues/173) [#175](https://github.com/hirosystems/ordhook/issues/175) [#178](https://github.com/hirosystems/ordhook/issues/178) * release develop ([#214](https://github.com/hirosystems/ordhook/issues/214)) ([4a31032](4a3103233b)) * release v1.0.2 ([#180](https://github.com/hirosystems/ordhook/issues/180)) ([ac3915f](ac3915f035)), closes [#170](https://github.com/hirosystems/ordhook/issues/170) [#171](https://github.com/hirosystems/ordhook/issues/171) [#285](https://github.com/hirosystems/ordhook/issues/285) [#310](https://github.com/hirosystems/ordhook/issues/310) [#168](https://github.com/hirosystems/ordhook/issues/168) [#173](https://github.com/hirosystems/ordhook/issues/173) [#175](https://github.com/hirosystems/ordhook/issues/175) [#178](https://github.com/hirosystems/ordhook/issues/178) * service boot sequence ([#175](https://github.com/hirosystems/ordhook/issues/175)) ([a744825](a74482588c)) commit4a3103233bAuthor: Ludo Galabru <ludo@hiro.so> Date: Thu Nov 23 07:24:07 2023 -0500 fix: release develop (#214) commitd0b57c5771Author: Ludo Galabru <ludo@hiro.so> Date: Thu Nov 23 07:18:06 2023 -0500 fix: databases lock commit2b49397848Merge:f82016934fc0d3Author: Ludo Galabru <ludo@hiro.so> Date: Thu Nov 23 07:17:01 2023 -0500 Merge branch 'main' into develop commitf820169aa0Author: Ludo Galabru <ludo@hiro.so> Date: Thu Nov 23 07:15:25 2023 -0500 fix: better handling of database locks (#200) commit977a30eb8dAuthor: omahs <73983677+omahs@users.noreply.github.com> Date: Thu Nov 23 13:06:00 2023 +0100 fix typos (#211) commit490fe01434Author: Ludo Galabru <ludo@hiro.so> Date: Thu Nov 16 19:48:14 2023 -0500 fix: testnet support (#208) commit0d2ff313d0Author: Ordinarius <137325073+ordinariusprof@users.noreply.github.com> Date: Thu Nov 16 08:30:23 2023 -0800 fix: around issue #187 for testnet (#204) * Update mod.rs * workaround for #187 * publish * build * revert * revert ci commit0714abf1b1Author: aravindgee <aravindgee@users.noreply.github.com> Date: Wed Nov 15 10:31:43 2023 -0500 CI: Enable CodeCov for Ordhook (#205) commit34fc0d362dMerge:47f365e977ec59Author: Ludo Galabru <ludo@hiro.so> Date: Wed Nov 1 20:55:07 2023 -0400 Merge branch 'develop' into main commit47f365eb47Author: Ludo Galabru <ludo@hiro.so> Date: Wed Nov 1 20:47:00 2023 -0400 fix: add destination to transfer events - release v1.0.2 * chore: stop auto-adding issues to DevTools Project (#170) * fix: enable streaming for in-memory observers (#171) * Squashed commit of the following: commit9862b71c34Author: semantic-release-bot <semantic-release-bot@martynus.net> Date: Thu Sep 7 00:06:39 2023 +0000 chore(release): 1.0.0 [skip ci] ## 1.0.0 (2023-09-07) ### Features * ability to control inclusion of inputs/outputs/proofs/witness ([daf5547](daf55476c9)) * ability to download hord.sqlite ([3dafa53](3dafa53ac0)) * ability to generate config ([9fda9d0](9fda9d0d34)) * ability to replay inscriptions ([f1adca9](f1adca9b0f)) * ability to resume ([6c7eaa3](6c7eaa3bee)) * ability to target blocks ([f6be49e](f6be49e24d)) * ability to tolerate corrupted data ([adb1b98](adb1b988a6)) * ability to track updates when scanning bitcoin (+refactor) ([9e54bff](9e54bfff35)) * ability to update stacks db from cli + fix caching logic ([3ea9f59](3ea9f597af)) * add command to check stacks db integrity ([322f473](322f47343c)) * add get block command to cli ([97de0b0](97de0b071b)) * add log, fix ordinal transfers scan ([c4202da](c4202dad2c)) * add logs ([473ddd0](473ddd0595)) * add metrics to `/ping` response of event observer server ([#297](https://github.com/hirosystems/ordhook/issues/297)) ([0e1ee7c](0e1ee7c1ee)), closes [#285](https://github.com/hirosystems/ordhook/issues/285) * add option to skip chainhook node ping ([a7c0b12](a7c0b12ad9)) * add options for logs ([917090b](917090b408)) * add post_transfer_output_value ([4ce0e9e](4ce0e9e5db)) * add retry ([117e41e](117e41eae8)) * add shared cache ([07523ae](07523aed1a)) * add support for bitcoin op DelegatedStacking ([6516155](6516155055)) * add transfers table ([db14f60](db14f60347)) * always try to initialize tables when starting service ([1a9eddb](1a9eddb6aa)) * attempt to scale up multithreading ([be91202](be91202d6b)) * attempt to support cursed inscriptions ([9b45f90](9b45f908b8)) * attempt transition to lazy model ([dda0b03](dda0b03ea3)) * batch ingestion, improve cleaning ([168162e](168162e0dd)) * better handling of blessed inscription turning cursed ([f11509a](f11509ab97)) * cascade changes in CLI interface ([24f27fe](24f27fea63)) * cascade hord activation ([42c090b](42c090ba7e)) * chainhook-sdk config niteties ([7d9e179](7d9e179464)) * class interface ([9dfec45](9dfec454f5)) * client draft ([6a6451c](6a6451c864)) * complete migration to lazy blocks ([fa50584](fa5058471a)) * disable certs ([389f77d](389f77d473)) * draft naive inscription detection ([9b3e38a](9b3e38a441)) * draft ordhook-sdk-js ([b264e72](b264e7281b)) * draft sha256 verification (wip) ([e6f0619](e6f0619a7c)) * drafting lazy deserialization ([eaa2f71](eaa2f71fce)) * dry config ([135297e](135297e978)) * expose `is_streaming_blocks` prop ([1ba27d7](1ba27d7459)) * expose more functions for working with the indexer ([654fead](654feadbdf)) * expose scanning status in GET endpoint ([156c463](156c463cc0)) * expose transfers_pre_inscription ([65afd77](65afd77492)) * fetch full bitcoin block, including witness data ([ee9a345](ee9a3452ac)) * fix download block ([38b50df](38b50df7a1)) * handle stacks unconfirmed state scans ([f6d050f](f6d050fbce)) * handle transfer ([fd5da52](fd5da52df4)) * HTTP responses adjustments ([51572ef](51572efd93)) * implement and document new development flow ([66019a0](66019a06e7)) * implement zmq runloop ([c6c1c0e](c6c1c0ecce)) * import inscription parser ([45e0147](45e0147ecf)) * improve cli ergonomics ([991e33f](991e33ff42)) * improve cli experience ([e865628](e8656285b2)) * improve debug log ([5df77d7](5df77d7f84)) * improve hord db commands ([21c09c2](21c09c296f)) * improve onboarding ([deaa739](deaa739bdd)) * improve ordinal scan efficiency ([e510d4b](e510d4bd09)) * improve README ([f30e6f4](f30e6f4ed5)) * improve repair command conveniency ([46be0ab](46be0ab5a7)) * improving curse approach ([dcb8054](dcb805485f)) * in-house thread pool ([bc5ffdd](bc5ffddb5b)) * inscription replay speedup ([33a4f8b](33a4f8b6af)) * introduce check command ([f17dc4c](f17dc4c343)) * introduce evaluation reports ([54ad874](54ad874ee5)) * introduce migration script ([8c2b16c](8c2b16cc48)) * introduce new predicate + refactor schemas ([611c79c](611c79cee3)) * introduce rocksdb storage for Stacks ([4564e88](4564e8818a)) * introduce sync command ([ab022e6](ab022e6098)) * introduce terminate function ([91616f6](91616f6531)) * is_streaming_blocks ([aacf487](aacf487de6)) * keep 1st tx in cache ([0978a5d](0978a5d4c1)) * logic to start ingestion during indexing ([3c1c99d](3c1c99df5d)) * merge "inscription_revealed" and "inscription_transferred" into "inscription_feed" ([741290d](741290de13)) * migrate stacks scans to rocksdb ([4408b1e](4408b1e7ec)) * migration to rocksdb, moving json parsing from networking thread ([5ad0147](5ad0147fa0)) * move thread pool size to config ([bc313fa](bc313fad5c)) * multithread traversals ([fba5c89](fba5c89a48)) * number of retries for 4 to 3 ([b294dff](b294dff69a)) * optimize memory ([5db1531](5db1531a3d)) * optimize replay ([be26dac](be26daccd0)) * ordinal inscription_transfer code complete ([f55a5ee](f55a5ee167)) * plug inscription processing in ibd ([df36617](df36617214)) * plumbing for ordhook-sdk-js ([7487589](74875896a3)) * polish `hord find sat_point` command ([d071484](d0714842a2)) * polish first impression ([3c2b00c](3c2b00ce38)) * predicate schemas ([198cdaa](198cdaa6c8)) * prototype warmup ([fa6c86f](fa6c86fb1f)) * re-approach stacks block commit schema ([218d599](218d5998d6)) * re-implement satoshi overflows handling ([8ea5bdf](8ea5bdf819)) * re-introduce ingestion ([71c90d7](71c90d755d)) * restore ability to replay transfers ([98e7e9b](98e7e9b21d)) * return enable in api ([f39259c](f39259ceeb)) * return local result when known ([5441851](5441851db7)) * revisit caching strategy ([2705b95](2705b9501b)) * revisit threading model ([05b6d5c](05b6d5c4d7)) * scan inscription revealed ([84c5a0c](84c5a0c521)) * scan inscription revealed ([644d515](644d5155d2)) * share traversals_cache over 10 blocks spans ([b0378c3](b0378c3099)) * simplify + improve coordination ([1922fd9](1922fd9bc4)) * start investigating zmq signaling ([0ec2653](0ec265380c)) * streamline processors ([13421db](13421db297)) * support cursed inscriptions in chainhook client ([d7cc5a4](d7cc5a4410)) * support for latest archives, add logs ([494cf3c](494cf3c9a5)) * tweak mmap / page_size values ([5316a57](5316a575b0)) * update chainhook-sdk ([f052e08](f052e08469)) * update inscription transfer logic ([9d0d106](9d0d106e9c)) * update inscription transfer schemas ([f80e983](f80e983481)) * upgrade `service start` implementation + documentation ([02db65e](02db65e417)) * use caching on streamed blocks ([784e9a0](784e9a0830)) * use thread pools for scans ([45b9abd](45b9abd3e0)) * zmq sockets ([d2e328a](d2e328aa57)) ### Bug Fixes * ability to run without redis ([96825c3](96825c35a8)) * add busy handler ([d712e0d](d712e0ddae)) * add exp backoff ([f014c14](f014c14277)) * add retry logic in rocksdb ([247df20](247df2088a)) * add retry logic to work around unexpected responses from bitcoind ([2ab6b32](2ab6b32ff0)) * additional adjustments ([fe26063](fe26063513)) * additional fixes (network, address, offsets) ([8006000](8006000034)) * address build warnings ([dc623a0](dc623a01e5)) * address non-inscribed block case ([a7d08a3](a7d08a3722)) * address redis disconnects ([a6b4a5f](a6b4a5fb38)) * address remaining issues ([74b2fa9](74b2fa9411)) * adjust error message ([3e7b0d0](3e7b0d03f9)) * allow empty block ([fe8ce45](fe8ce455a1)) * always fetch blocks ([97060a1](97060a13ca)) * async/await regression ([676aac1](676aac196d)) * attempt ([9e14fce](9e14fce0e4)) * attempt to fix offset ([e6c5d0e](e6c5d0eed8)) * attempt to retrieve blocks from iterator ([f718071](f718071b33)) * attempt to tweak rocksdb ([11b9b6b](11b9b6be62)) * auto enable stacks predicate ([30557f8](30557f8667)) * backpressure on traversals ([3177e22](3177e22921)) * batch inscription ([cd1085c](cd1085ceb0)) * batch migration ([ed8b7ad](ed8b7ad2f3)) * better redis error handling ([debb06c](debb06cd5c)) * better support of reinscriptions ([a1410e2](a1410e29dd)) * better termination ([8a5482c](8a5482c131)) * binary name ([4950a50](4950a50381)) * block streaming ([dcdfd16](dcdfd1655c)) * boot sequence ([577f1c2](577f1c237e)) * boot sequence, logs, format ([d03f851](d03f85178d)) * borrow issue ([66e2a7c](66e2a7c785)) * broken build ([f0d471e](f0d471ea8b)) * broken test ([239b26a](239b26a614)) * broken tests ([2ab6e7d](2ab6e7d679)) * build ([4067f08](4067f0814f)) * build ([607ac95](607ac953b1)) * build error ([d6ed108](d6ed10894c)) * build error ([bbede8b](bbede8b546)) * build error ([fa802fa](fa802fae7a)) * build error ([44ca74b](44ca74b2c5)) * build error ([053b781](053b7815a8)) * build error ([5c3bcf4](5c3bcf42fc)) * build error ([b78c0cc](b78c0ccea6)) * build error ([879ed67](879ed6775a)) * build errors ([60cd4d0](60cd4d0c61)) * build errors ([8dd91bf](8dd91bfce3)) * build errors / merge snafu ([47da0c1](47da0c132a)) * build errors + warnings ([938c6df](938c6dff27)) * build failing ([83f1496](83f14964a6)) * build warning ([561e51e](561e51eb27)) * build warning ([75847df](75847df0d1)) * build warning ([0194483](0194483b75)) * build warnings ([d3e998c](d3e998c469)) * build warnings ([e7ad175](e7ad175805)) * build warnings ([670bde6](670bde6379)) * bump incoming payload limit to 20mb ([7e15086](7e150861a4)) * cache invalidation ([05bd903](05bd9035eb)) * cache L2 capacity ([e2fbc73](e2fbc73eaf)) * cache size ([ce61205](ce61205b96)) * cache's ambitions ([e438db7](e438db7514)) * Cargo.toml ([759c3a3](759c3a393f)) * chain mixup, add logs ([0427a10](0427a10a63)) * change forking behavior ([4c10014](4c100147c2)) * clean expectations ([f9e089f](f9e089f90d)) * clear cache more regularly ([c3b884f](c3b884fd30)) * command for db patch ([27f6838](27f683818d)) * commands doc ([3485e6f](3485e6f3d9)) * compatibility with clarinet ([a282655](a28265509f)) * condition ([0233dc5](0233dc5bf0)) * create dummy inscription for sats overflow ([84aa6ce](84aa6ce7fd)) * db init command ([55e293b](55e293b3ca)) * decrease compression - from 4 bytes to 8 bytes ([b2eb314](b2eb31424b)) * deployer predicate wildcard ([05ca395](05ca395da1)) * disable sleep ([41ecace](41ecacee0e)) * disable steam scan when scanning past blocks ([e2949d2](e2949d213a)) * disambiguate inscription_output_value and inscription_fee ([9816cbb](9816cbb70a)) * do not panic ([a0fa1a9](a0fa1a9301)) * doc drift ([b595339](b595339024)) * docker build ([df39302](df39302616)) * docker file ([6ad5206](6ad52061eb)) * dockerfile ([73ad612](73ad612ea4)) * dockerfile ([da21ec4](da21ec4cb9)) * documentation drift ([c5335a7](c5335a765c)) * documentation drift ([38153ca](38153ca22f)) * don't early exit when satoshi computing fail ([a8d76b0](a8d76b03ac)) * don't enable predicate if error ([1274cbf](1274cbf9c4)) * early return ([8f97b56](8f97b5643b)) * edge case when requests processed in order ([8c4325f](8c4325f721)) * edge case when requests processed out of order ([a35cea2](a35cea2b54)) * edge case when requests processed out of order ([a6651b8](a6651b851f)) * enable profiling ([f99b073](f99b073528)) * enable specs on reboot ([f23be24](f23be246c2)) * enforce db reconnection in http endpoints ([bcd2a45](bcd2a45a86)) * enum serialization ([67cb340](67cb340674)) * error management ([f0274f5](f0274f5726)) * export all types on ts client ([be8bfbc](be8bfbcf60)) * failing build ([1502d5d](1502d5d682)) * fee ([0337f92](0337f92ce0)) * filter out sat overflows from payloads ([ce439ae](ce439ae900)) * gap in stacks scanning ([8c8c5c8](8c8c5c8611)) * generator typo ([8a7eddb](8a7eddb092)) * handle hint and case of re-inscriptions ([f86b184](f86b184832)) * handle non-spending transaction ([cb01eb5](cb01eb55fd)) * handle re-inscription for unbound inscriptions ([a1ffc1a](a1ffc1a59a)) * hard coded dev-dependency ([5c105de](5c105de8b5)) * ignore invalid inscription ([f18bc00](f18bc00f5a)) * ignore transaction aborting that we could not classify ([37c80f7](37c80f7e83)) * implement error handler ([d071b81](d071b81954)) * improve progress bar ([b28da56](b28da5697d)) * improve rewrite block command ([d524771](d52477142a)) * in-block re-inscription case ([90db9c3](90db9c3d15)) * include blocks discovered during scan, if any ([1eabce2](1eabce25c3)) * include ordinals operations in standardized blocks ([a13351d](a13351d46a)) * include proof on scan commands ([6574008](6574008ae8)) * increase number of retries ([343ddd6](343ddd65a8)) * indexing ([45661ab](45661ab62c)) * inject l1 cache hit in results (+ clearing) ([62fd929](62fd92948e)) * inscription fee ([2ac3022](2ac302235c)) * inscription_number ([a7d8153](a7d8153a8c)) * insert new locations ([6475aeb](6475aeb8d4)) * iterate on values ([0c73e62](0c73e62902)) * keep trying opening rocksdb conn if failing ([dbc794a](dbc794a0d4)) * lazy block approach ([b567322](b567322859)) * leader_registered doc ([f9d7370](f9d7370c43)) * loading predicates from redis ([3bd308f](3bd308fb15)) * log level, zeromq dependency ([4a2a6ef](4a2a6ef297)) * logic determining start height ([5dd300f](5dd300fb05)) * logs ([81be24e](81be24ef08)) * mark inscriber_address as nullable ([77fd88b](77fd88b9c1)) * more pessimism on retries ([9b987c5](9b987c51a9)) * move parsing back to network thread ([bad1ee6](bad1ee6d4e)) * moving stacks tip ([87c409e](87c409e01c)) * multithreading cap ([c80ae60](c80ae60991)) * myriad of improvements ([0633182](063318233d)) * nefarious logs ([3b01a48](3b01a48f1e)) * network, cascade changes ([1f45ec2](1f45ec26da)) * off by one ([2a0e75f](2a0e75f6a3)) * off by one ([c31611f](c31611fb28)) * off by one ([94e1141](94e11411f8)) * off by one ([abf70e7](abf70e7204)) * off by one error ([3832cf9](3832cf9770)) * off by one inscriptions number ([cdfbf48](cdfbf487fa)) * off by one isssue ([fead2ed](fead2ed693)) * off by one issue ([a8988ba](a8988ba573)) * off by one issue ([155e3a6](155e3a6d29)) * off by one issue on sats overflow ([8a12004](8a120040e7)) * off-by-one error in backward traversal ([d4128aa](d4128aa8a1)) * off-by-one in sats number resolution ([42acbeb](42acbebcd5)) * offset ([278a655](278a65524b)) * only avoid override for blessed inscriptions ([b50bbc1](b50bbc1bf7)) * optimize reg and dereg ([c2ec1b5](c2ec1b5283)) * ordinals scans ([62b62bd](62b62bd98a)) * outdated dockerfile ([771b036](771b0362b2)) * outdated documentation ([f472a49](f472a49c42)) * overriden inscriptions ([25c6441](25c6441404)) * parsing ([1f047a9](1f047a9162)) * patch absence of witness data ([f8fcfca](f8fcfcad6d)) * patch boot latency ([0e3faf9](0e3faf9a61)) * patch crach ([20d9df6](20d9df6c65)) * patch db call ([d385df2](d385df2037)) * pipeline logic ([a864c85](a864c85c33)) * pipeline resuming ([06883c6](06883c655a)) * ports ([3ee98a8](3ee98a8be9)) * potential resolve coinbase spent ([5d26738](5d267380f7)) * PoxInfo default for scan commands ([a00ccf5](a00ccf589a)) * predicate documentation ([572cf20](572cf202ba)) * predicate generator network ([8f0ae21](8f0ae216c8)) * provide optional values ([2cbf87e](2cbf87ebcc)) * re-apply initial fix ([f5cb516](f5cb516ee0)) * re-arrange logs ([2857d0a](2857d0a1a4)) * re-enable sleep ([0f61a26](0f61a26fda)) * re-initiate inscriptions connection every 250 blocks ([39671f4](39671f4378)) * re-qualify error to warn ([9431684](9431684afe)) * re-wire cmd ([a1447ad](a1447ad277)) * README ([db1d584](db1d584827)) * recreate db conn on a regular basis ([81d8575](81d85759a4)) * redis update ([d4889f1](d4889f16b7)) * related issue ([4b3a0da](4b3a0daa43)) * remove rocksdb reconnect ([f2b067e](f2b067e85e)) * remove sleep ([c371e74](c371e74de7)) * remove start logic ([a04711a](a04711ad7c)) * remove support for p2wsh inscription reveal support ([4fe71f2](4fe71f2622)) * remove symbols ([108117b](108117b82e)) * remove thread_max * 2 ([359c6f9](359c6f9422)) * reopen connect on failures ([3e15da5](3e15da5565)) * reply with 500 on payload processing error ([eaa6d7b](eaa6d7b640)) * report generation ([0dce12a](0dce12a4e2)) * restore stable values ([fb5c591](fb5c591943)) * return blocks to rollback in reverse order ([9fab5a3](9fab5a34a2)) * reuse existing computation for fix ([222f7c3](222f7c3a14)) * revert fix, avoid collision in traversals map ([dfcadec](dfcadec680)) * revisit log level ([4168661](416866123a)) * revisit transfer loop ([1f2151c](1f2151c098)) * rocket_okapi version ([2af31a8](2af31a8e64)) * safer db open, dockerfile ([43d37d7](43d37d73f2)) * safer error handling ([11509e4](11509e4435)) * sat offset computation ([b278b66](b278b66f84)) * sats overflow handling ([a3f745c](a3f745cfa7)) * schema for curse_type ([72d43c6](72d43c6b41)) * serialize handlers in one thread ([cdfc264](cdfc264cff)) * slow down initial configuration ([3096ad3](3096ad3b26)) * sql query ([1a3bc42](1a3bc428ea)) * sql query bis ([a479884](a4798848b1)) * sql request ([6345df2](6345df2652)) * sql table setup ([c8884a7](c8884a7dbe)) * stack overflow ([aed7d5d](aed7d5d005)) * stacks predicate format ([fcf9fb0](fcf9fb0e3f)) * start_block off by one ([b99f7b0](b99f7b0011)) * streamline txid handling ([ad48351](ad48351044)) * test suite ([c7672f9](c7672f91a1)) * test warns and errors ([0887d6b](0887d6b8ca)) * threading model ([c9c43ae](c9c43ae3e3)) * threading model ([c2354fc](c2354fcacd)) * track interrupted scans ([2b51dca](2b51dca8f3)) * transaction type schema ([c35a737](c35a737ed2)) * transfer recomputing commit ([3643636](364363680f)) * transfer tracking ([0ea85e3](0ea85e3d20)) * transfer tracking ([30f299e](30f299ef7c)) * transfer tracking ([0cd29f5](0cd29f5925)) * transfer tracking + empty blocks ([dc94875](dc948755b2)) * traversals algo ([e8ee3ab](e8ee3ab036)) * tweak rocksdb options ([a0a6950](a0a69502d8)) * typo ([b0498bb](b0498bb048)) * typo ([baa773f](baa773ff4d)) * unexpected expectation ([7dd362b](7dd362b4f5)) * unify rosetta operation schemas ([bf3216b](bf3216b100)) * unused imports ([3aab402](3aab4022ab)) * update chainhook schema ([4e82714](4e8271491b)) * update inscription_number ([89b94e7](89b94e7d5d)) * update license ([6ebeb77](6ebeb77d6a)) * update rust version in docker build ([fab6f69](fab6f69df5)) * update spec status ([e268925](e2689255b7)) * update/pin dependencies ([#311](https://github.com/hirosystems/ordhook/issues/311)) ([f54b374](f54b374b24)), closes [#310](https://github.com/hirosystems/ordhook/issues/310) * use first input to stick with ord spec interpretation / implementation ([206678f](206678f0d1)) * use rpc instead of rest ([1b18818](1b188182f1)) * zeromq, subsidy issue ([dbca70c](dbca70c197)) ### Reverts * Revert "chore: tmp patch" ([3e022ca](3e022ca322)) commit4ef18d5b1eMerge:d111c444cde5e8Author: Scott McClellan <scott.mcclellan@gmail.com> Date: Wed Sep 6 18:44:26 2023 -0500 Merge pull request #168 from hirosystems/develop Merge up `develop` to `main` * fix: CI rust version mismatch, create empty db (#173) * fix: create db if does not exists * chore: update rust version * chore: bump version to 1.0.1 * fix: service boot sequence (#175) * fix: ci * fix: initial flow (#178) * chore: update chainhook-sdk + cascade changes * fix: update archive url * feat: only create rocksdb if sqlite present * fix: use crossbeam channel instead of std * fix: improve error message * doc: update README * fix: build warnings * fix: block_archiving expiration * fix: archive url * fix: read content len from http header * chore: untar sqlite file * chore: bump versions * fix: build error / warning * change title (#182) * Ordhook doc updates (#183) * update copy * add openapi reference file to ordhook docs for better context * typo in reference link for bitcoind * remove references to chainhook * break out guides for scanning and streaming ordinal activities * fix references to Ordhook.toml * update content for each guide * replace mentions of Chainhook --------- Co-authored-by: Max Efremov <51917427+mefrem@users.noreply.github.com> * provide a pointer to rust installation and a next steps linking to guides (#184) * update Ordhook.toml (#185) * fix: grammar tweaks grammar tweaks * fix: grammar tweaks Grammar tweaks * fix: grammar updates grammar updates Co-authored-by: Ludo Galabru <ludo@hiro.so> * doc: update getting-started Co-authored-by: Ludo Galabru <ludo@hiro.so> * doc: update overview.md Updated grammar Co-authored-by: Ludo Galabru <ludo@hiro.so> * feat: ordhook-sdk-js refactoring (#186) --------- Co-authored-by: Scott McClellan <scott.mcclellan@gmail.com> Co-authored-by: Ryan <ryan.waits@gmail.com> Co-authored-by: Max Efremov <51917427+mefrem@users.noreply.github.com> Co-authored-by: max-crawford <102705427+max-crawford@users.noreply.github.com> * chore: update Cargo.lock * chore: cargo fmt * fix: merge artifact * fix: rocksdb fine tuning * fix: Cargo.toml warns * feat: auto-repair at boot * fix: rocksdb conn handling * fix: improve backoff strategy * chore: refactor BlockBytesCursor usage * fix: update sequence_metadata when augmenting block * feat: revisit observers handling # Conflicts: # components/ordhook-core/src/service/mod.rs * fix: aborted loop * fix: stateful observers * fix: build warnings * chore: tweak db config * fix: include coinbase output offset * chore: tweak cli * fix: iterator usage * chore: disable blobdb * fix: broken test * chore: update dependencies * chore: augment db schema * chore: update ord * chore: inscription parsing re-integration * feat: persist inscription_subindex * feat: update sequence_metadata * fix: sql field name * fix: build warnings * chore: update chainhook sdk * fix: indexing subindex * fix: tx chaining reinscription handling * chore: add missing indexes * chore: restore set_max_open_files * fix: dockerfile * feat: panic when unable to retrieve ordinal number * chore: disable integrity check for now * chore: exit process on error * fix: attempt to fix race condition * chore: disable cache tmp * fix: patch traversals can be reused * fix: patch traversals can be reused * chore: fix expectations * fix: replay * bump: major version
/ / ▶ Ordhook
/ --- / Ordinal indexing engine based on Chainhook.
/ / Build indexes, standards and protocols on top of Ordinals and Inscriptions (BRC20, etc).
Introduction
The Ordinal Theory is a protocol aiming at attributing unique identifiers to minted satoshis (sats). With its numbering scheme, satoshis can be inscribed with arbitrary content (aka inscriptions), creating bitcoin-native digital artifacts more commonly known as NFTs. Inscriptions do not require a sidechain or separate token, which makes it attractive for new entrants to adopt, extend, and use. These inscribed sats can be transferred using Bitcoin transactions, sent to Bitcoin addresses, and held in Bitcoin UTXOs. In all respects, these transactions, addresses, and UTXOs are normal Bitcoin transactions, addresses, and UTXOs, except that to send individual sats, transactions must control the order and value of inputs and outputs per Ordinal Theory.
Now that we discussed Ordinal Theory, let's dive into what ordhook attempts to solve for developers.
The ordhook is an indexer designed to help developers build new re-org-resistant applications on top of the Ordinal Theory. This indexer will make it easy for protocol developers and users of those protocols to trace and discover the ownership of Ordinal's inscriptions, along with a wealth of information about each inscription.
The ordhook uses Chainhook SDK from the Chainhook project, which is a re-org-aware transaction indexing engine for Stacks and Bitcoin. The SDK is designed with first-class event-driven principles, so it helps developers extract transactions from blocks efficiently and keeps a consistent view of the chain state.
With ordhook, Bitcoin developers can reliably implement feature-rich protocols and business models utilizing near-real-time Ordinals inscriptions and transfers events.
Quick Start
Installing ordhook from source
$ git clone https://github.com/hirosystems/ordhook.git
$ cd ordhook
$ cargo ordhook-install
Getting started with ordhook
Explore Ordinal activities in your terminal
Once ordhook is installed, Ordinals activities scanning can simply be performed using the following command:
$ ordhook scan blocks --interval 767430:767753 --mainnet
Inscription 6fb976ab49dcec017f1e201e84395983204ae1a7c2abf7ced0a85d692e442799i0 revealed at block #767430 (ordinal_number 1252201400444387, inscription_number 0)
Inscription 26482871f33f1051f450f2da9af275794c0b5f1c61ebf35e4467fb42c2813403i0 revealed at block #767753 (ordinal_number 727624168684699, inscription_number 1)
In this command, an interval of blocks to scan (starting at block 767430, ending at block 767753) is being provided. ordhook will display inscriptions and transfers activities occurring in the range of the specified blocks.
The activity for a given inscription can be retrieved using the following command:
$ ordhook scan inscription 6fb976ab49dcec017f1e201e84395983204ae1a7c2abf7ced0a85d692e442799i0 --mainnet
Inscription 6fb976ab49dcec017f1e201e84395983204ae1a7c2abf7ced0a85d692e442799i0 revealed at block #767430 (ordinal_number 1252201400444387, inscription_number 0)
Transferred in transaction bc4c30829a9564c0d58e6287195622b53ced54a25711d1b86be7cd3a70ef61ed at block 785396
Stream Ordinal activities to an indexer
ordhook is designed to help developers extract ordinals activities (inscriptions and transfers) from the Bitcoin chain and streaming these activities to their indexer / web application.
In order to get started, a bitcoind instance with access to the RPC methods getblockhash and getblock must be running. The RPC calls latency will directly impact the speed of the scans.
Note: the configuration of a bitcoind instance is out of scope for this guide.
Assuming:
1) a bitcoind node correctly configured and
2) a local HTTP server running on port 3000 exposing a POST /api/events endpoint,
A configuration file Ordhook.toml can be generated using the command:
$ ordhook config new --mainnet
✔ Generated config file Ordhook.toml
After adjusting the Ordhook.toml settings to make them match the bitcoind configuration, the following command can be ran:
$ ordhook scan blocks --interval 767430:767753 --post-to=http://localhost:3000/api/events --config-path=./Ordhook.toml
ordhook will retrieve the full Ordinals activities (including the inscriptions content) and send all these informations to the http://localhost:3000/api/events HTTP POST endpoint.
Run ordhook as a service for streaming blocks
ordhook can be ran as a service for streaming and processing new blocks appended to the Bitcoin blockchain.
$ ordhook service start --post-to=http://localhost:3000/api/events --config-path=./Ordhook.toml
New http-post endpoints can also be added dynamically by adding the following section in the Ordhook.toml configuration file:
[http_api]
http_port = 20456
Running ordhook with the command
$ ordhook service start --config-path=./Ordhook.toml
will spin up a HTTP API for managing events destinations.
A comprehensive OpenAPI specification explaining how to interact with this HTTP REST API can be found here.
Troubleshooting: Performance and System Requirements
The Ordinals Theory protocol is resource-intensive, demanding significant CPU, memory, and disk capabilities. As we continue to refine and optimize, keep in mind the following system requirements and recommendations to ensure optimal performance:
CPU: The ordhook tool efficiently utilizes multiple cores when detected at runtime, parallelizing tasks to boost performance.
Memory: A minimum of 16GB RAM is recommended.
Disk: To enhance I/O performance, SSD or NVMe storage is suggested.
OS Requirements: Ensure your system allows for a minimum of 4096 open file descriptors. Configuration may vary based on your operating system. On certain systems, this can be adjusted using the ulimit command or the launchctl limit command.