kylefang/stacks-puppet-node
1
0
Fork 0
mirror of https://github.com/alexgo-io/stacks-puppet-node.git synced 2026-03-28 23:58:29 +08:00
Code Issues Packages Projects Releases Wiki Activity

reorganization: pull docs, pkg, tools from blockstack_cli

Browse Source
This commit is contained in:
Jude Nelson
2017-01-17 11:17:23 -05:00
parent 38674b74c6
commit efb452c459
7 changed files with 1507 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

56
docs/README.md Normal file
View File

@@ -0,0 +1,56 @@
# Documentation
Blockstack CLI is both a command-line interface tool, a system service (daemon), and a Python library for interacting with Blockstack.
## Quick Links
* [Basic command-line usage](https://github.com/blockstack/blockstack-cli/blob/master/docs/basic_usage.md)
* [Advanced command-line usage](https://github.com/blockstack/blockstack-cli/blob/master/docs/advanced_usage.md)
* [Glossary of Terms](https://github.com/blockstack/blockstack-cli/blob/master/docs/glossary.md)
* [How to link your OpenBazaar Store to your Blockstack ID](https://github.com/blockstack/blockstack-cli/blob/master/docs/openbazaar.md)
## Architecture Overview
Most of the complexity of Blockstack lives in its client library. Specifically, the library does the following:
* Generating and sending name operation transactions.
* Reading, writing, and deleting data in your storage providers (and reading other peoples' data from their storage providers).
* Handling data authenticity, encryption, and validation.
* Querying a Blockstack Server for blockchain-hosted information.
The CLI tool is a wrapper around the library. Most of its commands are thin wrappers around library functions.
In addition to a CLI tool and library, Blockstack CLI comes with a system service that runs in the background as a daemon. The daemon does the following:
* Acts as a personal registrar. It queues up all your name operation transactions, waits for them to be confirmed by the blockchain, sends them out, and replicates your zonefile and profile as needed.
* Hosts your wallet. Your wallet is never stored in plaintext; it only lives in the daemon's RAM (for when it needs to send out transactions).
* Allows programmatic access to a subset of CLI commands. This allows other programs on your computer to do things like look up Blockstack IDs, query their data, and so on. For security, the daemon will never serve the wallet via the API, nor does it expose any API call that can change data or send transactions (it is effectively a read-only API).
## Files
These files are created by Blockstack CLI:
### Files You Can Edit
These files define how the CLI behaves.
* `~/.blockstack/client.ini`: This is the CLI config file. You can interactively modify it with `blockstack configure`.
* `~/.blockstack/wallet.json`: This is your JSON-encoded wallet. It contains your password-encrypted keys.
### Files You Can Read
These files are useful primarily for troubleshooting.
* `~/.blockstack/api_endpoint.log`: This is the log for the system service that manages your wallet, sends your name operations, and accesses your profile and data. It's a great source for troubleshooting.
* `~/.blockstack/api_endpoint.pid`: This contains the PID of the system service.
* `~/.blockstack/metadata/`: This directory contains versioning information for mutable data from other profiles you have read. It gets used to stop malicious storage providers from serving you older versions of the data you've already seen.
* `~/.blockstack/storage-disk/`: If you use the `disk` storage driver (it is activated by default), then this is where it holds your zonefiles, profiles, and data.
### Files You Should NOT Edit
You shouldn't touch these files unless you're a developer, and even then, you should only do so at your own risk.
* `~/.blockstack/queues.db`: This is a SQLite database that contains queued-up transactions for name operations. If you want to remove any stuck transactions, use `blockstack unqueue` in the [advanced](https://github.com/blockstack/blockstack-cli/blob/master/docs/advanced_usage.md) usage.
* `~/.blockstack/registrar.lock`: This is a lockfile held by a thread in the registrar thread in the CLI daemon.
* `~/.blockstack/blockchain_headers.dat`: SPV headers.
* `~/.blockstack/client.uuid`: Used for anonymous statistics gathering purposes (which you can disable using `blockstack configure`). If you remove it, a new one will be generated.

124
docs/advanced_usage.md Normal file
View File

@@ -0,0 +1,124 @@
# Advanced Usage
This section details some of the advanced features in the CLI.
## A Word of Warning
Advanced features are meant to be used by experienced Blockstack users and developers, They receive less UI/UX testing than basic features, and their interfaces are expected to change to accomodate bugfixes and security fixes. Moreover, improper use of some advanced methods can cost you money, corrupt your profile, or compromise your wallet. Once they receive sufficient testing, an advanced feature may become a basic-mode feature in a subsequent release.
**Do not use advanced mode unless you know what you are doing!**
## Activating Advanced Mode
To activate advanced mode, use the command `blockstack set_advanced_mode on`.
To deactivate it later (recommended), use the command `blockstack set_advanced_mode off`.
## Changing or Using Exiting Keys
If you already have a payment key you want to use, or an owner key you want to migrate over, you can generate a wallet directly with `import_wallet`. We recommend using this command interactively, so you know which keys correspond to which usages.
## Accounts
With the accounts methods, you can directly manage your social proofs, link existing services to your profile, and store small bits of information.
The account management methods are:
* `get_account`: Look up an account in a name's profile. There can be more than one match.
* `list_accounts`: List all accounts in a name's profile.
* `put_account`: Add or update an account in a name's profile.
* `delete_account`: Remove an account from a name's profile. This may need to be done more than once, if there are duplicates of the account.
## Advanced Blockstack ID Queries
Beyond `lookup` and `whois`, there are a few other more advanced queries you can run on Blockstack IDs. These include:
### Listing Blockstack IDs
* `get_all_names`: Get the list of every single Blockstack ID in existance.
* `get_names_owned_by_address`: Get the list of names owned by a particular ownership address.
### Querying the Blockchain
* `get_name_blockchain_record`: Get the raw database record for a Blockstack ID. It will contain a *compressed* history of all name operations that have affected it. This is meant primarily for debugging purposes; to get an easier-to-parse listing of the information this command returns, use `get_name_blockchain_history`.
* `get_name_blockchain_history`: Get the set of all prior states a Blockstack ID has been in, keyed by the block heights at which the state-change was processed.
* `get_records_at`: Get the list of name operation records processed at a particular block height.
* `list_update_history`: Get the list of all zonefile hashes that a Blockstack ID has ever had.
### Zonefiles
* `get_name_zonefile`: Get only a Blockstack ID's zonefile.
* `list_zonefile_history`: Get the list of all zonefiles a Blockstack ID has ever had. **NOTE:** There is no guarantee that the server will hold copies of old zonefiles. This command is meant mainly for determining which historic zonefiles a server has processed.
* `set_zonefile_hash`: This is the counterpart to `update`, but instead of setting the zonefile directly and uploading it to storage, you can use this command to directly set the data hash for a Blockstack ID. **NOTE:** You should ensure that the associated zonefile data has been replicated off-chain to a place where other users can get at it.
### Lightweight Queries
The lightweight lookup protocol for Blockstack is called *Simplified Name Verification* (SNV). This command returns a prior blockchain-level record given a more recent known-good consensus hash, serial number, or transaction ID of a transaction that contains a consensus hash. The CLI does not need to trust the Blockstack server to use these commands.
* `lookup_snv`: Use the Merkle skip-list in the name database to look up a historic name operation on a Blockstack ID.
## Consensus Queries
You can query consensus hash information from the server with the following commands:
* `consensus`: Get the consensus hash at a particular block height
## Namespace Queries
In addition to querying Blockstack IDs, the CLI has advanced commands for querying namespaces. These include:
* `get_namespace_blockchain_record`: Get the raw database record for a Blockstack namespace. It will contain a *compressed* history of all namespace operations that have affected it.
* `get_names_in_namespace`: Get the list of every Blockstack ID in a particular namespace.
* `get_namespace_cost`: Get the cost required to preorder a namespace. Does *not* include the cost to reveal and ready it, nor does it include the transaction fees.
## Namespace Creation
**WARNING:** We do not recommend that you try to do this by yourself. Creating a namespace is **EXTREMELY EXPENSIVE**. If you are interested in creating your own namespace, please contact the Blockstack developers on the [Blockstack Slack](http://chat.blockstack.org).
These methods allow you to create a namespace. There are three steps: preordering, revealing, and readying. Preordering a namespace is like preordering a name--you announce the hash of the namespace ID and the address that will control it. Revealing a namespace not only reveals the namespace ID, but also sets the pricing and lifetime rules for names in the namespace. After revealing the namespace, the namespace controller can pre-populate the namespace by importing Blockstack IDs. Once the namespace has been pre-populated, the controller sends a final transaction that readies the namespace for general use.
* `namespace_preorder`: Preorder a namespace.
* `namespace_reveal`: Reveal a namespace, and set its pricing and lifetime parameters. **NOTE:** This must be done within 144 blocks of sending the namespace preorder transaction.
* `name_import`: Import a name into a revealed (but not readied) namespace. You can set its owner address and zonefile hash directly.
* `namespace_ready`: Open a namespace for general registrations.
## Data Storage
Blockstack allows users to store arbitrary data to any set of storage providers for which the CLI has a driver. The data will be signed by the user's data key, so when other users read the data later on, they can verify that it is authentic (i.e. the storage provider is not trusted). Moreover, Blockstack is designed such that users don't have to know or care about which storage providers were used--as far as users can see, storage providers are just shared hard drives.
There are two types of data supported by Blockstack: *mutable* data, and *immutable* data. Mutable data is linked by the profile, and can be written as fast and as frequently as the storage provider allows. Mutable data is addressed by URL.
**WARNING:** While mutable data guarantees end-to-end authenticity, there is a chance that a malicious storage provider can serve new readers stale versions of the data. That is, users who have read the latest data already will not get tricked into reading stale data, but users who have *not yet* read the latest data *can* be tricked (i.e. the CLI keeps a version number for mutable data to do so). This must be taken into account if you intend to use this API.
Immutable data, however, is content-addressed, and its cryptographic hash is stored to the user's zonefile. Writing immutable data will entail updating the zonefile and sending an `update` transaction (handled internally), so it will be slow by comparison. This has the advantage that storage providers cannot perform the aforementioned stale data attack, but has the downside that writes cost money and take a long time to complete.
That said, we recommend using the mutable data API with several different storage providers whenever possible.
### Mutable Data
The following commands affect mutable data:
* `get_mutable`: Use the profile to look up and fetch a piece of mutable data.
* `put_mutable`: Add a link to mutable data to the profile, and replicate the signed data itself to all storage providers. Other users will need the data's name to read it with `get_mutable`.
* `delete_mutable`: Remove a link to mutable data from the profile, and ask all storage providers to delete the signed data.
### Immutable Data
The following commnds affect immutable data:
* `get_immutable`: Look up and fetch a piece of immutable data. You can supply either the name of the data, or its hash (both are stored in the zonefile, so there is no gain or loss of security in this choice).
* `put_immutable`: Replicate a piece of data to all storage providers, add its name and hash to the zonefile, and issue an `update` to upload the new zonefile to Blockstack servers and write the hash to the blockchain.
* `delete_immutable`: Remove the link to the data from the zonefile, ask all storage providers to delete the data, and issue an `update` to upload the new zonefile to Blockstack servers and write the new hash to the blockchain.
* `list_immutable_data_history`: Given the name of a piece of immutable data, query the zonefile history to find the historic list of hashes it has had. **NOTE:** Like `list_zonefile_history` above, this only returns data hashes for the data if the Blockstack server has the historic zonefile.
## Fault Recovery
Sometimes, things beyond our control can happen. Transactions can get stuck, storage providers can go offline or corrupt data, and so on. These commands are meant to assist in recovering from these problems:
* `set_profile`: Directly set a Blockstack ID's profile. All previous accounts, data links, etc. must be included in the new profile, since the old profile (if still present) will be overwritten by the one given here.
* `convert_legacy_profile`: Given a legacy profile taken from a resolver, directly convert it into a new profile. This can be used with `set_profile` to recover from a failed profile migration.
* `unqueue`: If a transaction gets lost or stuck, you can remove it from the CLI's transaction queue with this command. This will allow you to re-try it.
* `rpcctl`: This lets you directly start or stop the Blockstack CLI's background daemon, which lets you recover from any crashes it experiences (you can find a trace of its behavior in `~/.blockstack/api_endpoint.log`)
## Programmatic Access
Other programs may want to make RPC calls the Blockstack CLI daemon. They can do so using either the `blockstack_client` Python package, or they can do so via the CLI as follows:
* `rpc`: Issue a JSON RPC call. Takes a raw JSON string that encodes a list of arguments.

324
docs/basic_usage.md Normal file
View File

@@ -0,0 +1,324 @@
# Basic Usage
This section describes the basic features of Blockstack CLI. This document is meant to supplement the built-in documentation in the tool.
If at any point you forget how to use a particular CLI command, you can just type `blockstack <NAME OF COMMAND>`, and you will be interactively prompted for each piece of data it needs.
## Client Setup
By default, the CLI tool will automatically configure itself with sensible defaults. If you don't like the defaults, you can change them by running `blockstack configure`.
In particular, you can enable the advanced features with `blockstack set_advanced_mode on`. Please see the `advanced_usage.md` file for details.
## Wallet Setup
Blockstack uses its own wallet, stored to `~/.blockstack/wallet.json`. You do not need a wallet to do lookups.
Blockstack will set up a wallet for you when you try to do something that requires payment (e.g. registering a name). When prompted, you will be asked for a wallet password. This password will be used to encrypt your private keys, and must be at least 16 characters. We recommend using a random string of words that you can easily memorize, since **there is no way to recover your wallet if you forget your password.**
Once you generate a wallet, **make a back-up to a USB key**. You can also print it out, since it's just a string of plain text.
## Doing Lookups
There are two kinds of lookups: a `whois` and a `lookup`. The `whois` queries only information found in the blockchain, whereas a `lookup` queries off-chain data.
A `lookup` fetches the off-chain profile and zonefile. For example:
```
$ blockstack lookup guylepage3.id
{
"profile": {
"@type": "Person",
"account": [
{
"@type": "Account",
"identifier": "1Mp5vKwCbekeWetMHLKDD2fDLJzw4vKxiQ",
"role": "payment",
"service": "bitcoin"
},
{
"@type": "Account",
"identifier": "guylepage3",
"proofType": "http",
"proofUrl": "https://twitter.com/guylepage3/status/731252874886385665",
"service": "twitter"
},
{
"@type": "Account",
"identifier": "g3lepage",
"proofType": "http",
"proofUrl": "https://www.facebook.com/g3lepage/posts/10154223148908760",
"service": "facebook"
},
{
"@type": "Account",
"identifier": "guylepage3",
"proofType": "http",
"proofUrl": "https://gist.github.com/guylepage3/06f522444fb71f1daf01a534396d1f9e",
"service": "github"
}
],
"address": {
"@type": "PostalAddress",
"addressLocality": "New York, NY"
},
"description": "@blockstackorg developer. 1st hire, Design Partner @blockstacklabs (YC/USV backed) entrepreneur, blockchain, creative, marketing, surf, triathlon, ironman",
"graph": {
"url": "https://s3.amazonaws.com/grph/guylepage3"
},
"image": [
{
"@type": "ImageObject",
"contentUrl": "https://s3.amazonaws.com/dx3/guylepage3",
"name": "cover"
},
{
"@type": "ImageObject",
"contentUrl": "https://s3.amazonaws.com/kd4/guylepage3",
"name": "avatar"
}
],
"name": "Guy Lepage",
"website": [
{
"@type": "WebSite",
"url": "http://blockstack.com/team"
}
]
},
"zonefile": {
"$origin": "guylepage3.id",
"$ttl": 3600,
"uri": [
{
"name": "_http._tcp",
"priority": 10,
"target": "https://blockstack.s3.amazonaws.com/guylepage3.id",
"weight": 1
}
]
}
}
```
There will always be two keys defined: `profile` and `zonefile`. The hash of the contents of the `zonefile` object are stored in the blockchain, and the Blockstack servers hold copies of zonefile data. The `profile` object is constructed from a **signed JSON web token** (JWT), which is hosted on the storage providers of the user's choice (as determined in the `client.ini` config file). A modern zonefile points to where the profile JWT is hosted.
The `whois` query is more low-level, and pulls up information that's hosted only in the blockchain. Here's a sample `whois` query:
```
$ blockstack whois muneeb.id
{
"approx_expiration_date": "2016 Sep 09 13:12:31 UTC",
"block_preordered_at": 373821,
"block_renewed_at": 373821,
"expire_block": 426416,
"has_zonefile": true,
"last_transaction_height": 402804,
"last_transaction_id": "904c5f187ab143d187e26afaddaa6061059451407193fbfc4c4a9b0baa24dbd7",
"owner_address": "1QJQxDas5JhdiXhEbNS14iNjr8auFT96GP",
"owner_script": "76a914ff95f5612a26a81e919e4b6e63fdd929fd115d6d88ac",
"zonefile_hash": "3085137b19ce56092f5cb91b7f78d073c815dbc1"
}
```
This information includes the block heights at which the name was preordered (for the first time ever), when it was registered to its curent owner, when it will expire, and when the last transaction (whose ID is given) that affected the name was seen. It also encodes the hash of the zonefile (stored off-chain), the owner's address (public key hash), and owner script (scriptPubKey from Bitcoin).
Note that the `approx_expiration_data` is *approxmiate*. The system uses the `expire_block` to determine when exactly the name expires; the date is extrapolated from the average block time. You should renew your name well before it expires, just to be sure the transaction gets accepted. We recommend doing it 1,000 blocks before it is set to expire.
### A Note on Legacy Profiles
Older profiles used a different structure for storing information. Looking them up will produce a "legacy" zonefile, as well as the profile it represents.
Legacy zonefiles do not look like DNS zonefiles at all. For example:
```
$ blockstack lookup muneeb.id
{
"profile": {
"@type": "Person",
"account": [
{
"@type": "Account",
"identifier": "muneeb",
"proofType": "http",
"service": "twitter"
},
{
"@type": "Account",
"identifier": "muneeb.ali",
"proofType": "http",
"service": "facebook"
},
{
"@type": "Account",
"identifier": "muneeb-ali",
"proofType": "http",
"service": "github"
},
{
"@type": "Account",
"identifier": "1LNLCwtigWAvLkNakUK4jnmmvdVvmULeES",
"role": "payment",
"service": "bitcoin"
},
{
"@type": "Account",
"contentUrl": "http://muneebali.com/static/files/key.asc",
"identifier": "9862A3FB338BE9EB6C6A5E05639C89272AFEC540",
"role": "key",
"service": "pgp"
}
],
"address": {
"@type": "PostalAddress",
"addressLocality": "New York, NY"
},
"description": "Co-founder of Onename (YC S14), final-year PhD candidate at Princeton. Interested in distributed systems and blockchains.",
"image": [
{
"@type": "ImageObject",
"contentUrl": "https://s3.amazonaws.com/kd4/muneeb",
"name": "avatar"
},
{
"@type": "ImageObject",
"contentUrl": "https://s3.amazonaws.com/dx3/muneeb",
"name": "cover"
}
],
"name": "Muneeb Ali",
"website": [
{
"@type": "WebSite",
"url": "http://muneebali.com"
}
]
},
"zonefile": {
"avatar": {
"url": "https://s3.amazonaws.com/kd4/muneeb"
},
"bio": "Co-founder of Onename (YC S14), final-year PhD candidate at Princeton. Interested in distributed systems and blockchains.",
"bitcoin": {
"address": "1LNLCwtigWAvLkNakUK4jnmmvdVvmULeES"
},
"cover": {
"url": "https://s3.amazonaws.com/dx3/muneeb"
},
"facebook": {
"proof": {
"url": "https://facebook.com/muneeb.ali/posts/10152524743274123"
},
"username": "muneeb.ali"
},
"github": {
"proof": {
"url": "https://gist.github.com/muneeb-ali/0f00d4da967646ee0bc3"
},
"username": "muneeb-ali"
},
"graph": {
"followee_count": 4,
"url": "https://s3.amazonaws.com/grph/muneeb"
},
"location": {
"formatted": "New York, NY"
},
"name": {
"formatted": "Muneeb Ali"
},
"pgp": {
"fingerprint": "9862A3FB338BE9EB6C6A5E05639C89272AFEC540",
"url": "http://muneebali.com/static/files/key.asc"
},
"twitter": {
"proof": {
"url": "https://twitter.com/muneeb/status/483765788478689280"
},
"username": "muneeb"
},
"v": "0.2",
"website": "http://muneebali.com"
}
}
```
## Blockstack Wallet
The Blockstack wallet has three keys: your payment key, your ownership key (i.e. the key that owns the names), and your data key (i.e. the key that signs your profile data).
In the basic mode of operation, you can query information about them with these commands:
* `blockstack balance`: Query your payment account balance (excludes transactions with less than 6 confirmations).
* `blockstack deposit`: Get your payment address information. This is the address that **pays for names and transaction fees**.
* `blockstack import`: Get your name owner address. This is the address for **transferring a name to a different wallet**.
* `blockstack names`: See the list of names that are owned by your owner address.
## Registering a Name
To register a name, simply type:
```
$ blockstack register <YOUR NAME>.id
```
At this time, the name must end in `.id`. It will be registered in the `.id` namespace, since only the `.id` namespace exists (see [here](https://blockstack.org/docs/namespaces) for details).
You will be prompted to confirm the purchase, and (if you haven't entered it yet), you will be prompted for your wallet password.
If you'd like to see the price of a name, without actually purchasing it, you can use `blockstack price <YOUR NAME>.id`.
### A Note on Transaction Fees
The total name cost includes all the relevant transaction fees. However, fee prices are dynamic, and may change during the registration (which requires issuing three transactions).
To ensure timely registration, you should fill your payment address with **at least +0.001 BTC more than the name cost**.
## Migrating an Existing Name to the New Profile Schema
Some advanced options are disabled for older names registered through [Onename](https://onename.com). To enable them, you will need to migrate your name's off-chain data.
To do so, run `blockstack migrate <YOUR NAME>.id`. It will take about an hour and a half to complete, but only needs to be done once, and only if your name has a legacy zonefile (see above).
## Checking the Blockstack Server
You can track the progress of your name transactions with `blockstack info`, which will show you which names have unconfirmed transactions (and what kind they are). The CLI waits for 10 confirmations before considering it confirmed.
When registering a name, a name will pass through three states: `preorder`, `register`, and `update`. The first step registers your public key and the hash of the name, and waits for it to be confirmed (so no one can front-run you when you reveal the name). The second step reveals the name in the blockchain; you can look at the transaction in a block explorer and find it in the `OP_RETURN` data. The third stage sets up your zonefile and your profile, and writes your zonefile's hash to the blockchain.
## Other Operations
### Transferring a Name
You can send a name to a new ownership address with `blockstack transfer <YOUR NAME>.id <NEW ADDRESS>`.
### Renewing a Name
Names do not last forever, and must be periodically renewed. You can see when a name expires using `whois`, and renew it with `blockstack renew <YOUR NAME>.id`.
**If you do not renew your name, someone else can register it, and you will not be able to get it back.** The best alternative is to try to ask the new owner if (s)he will sell it back to you.
### Revoking a Name
If you lose your Blockstack wallet or the device(s) that host it, you have the option of revoking the name using a backed up copy of your wallet. To do so, type `blockstack revoke <YOUR NAME>.id`.
### Updating a Name's Zonefile
**OpenBazaar Users**: If you are trying to add your OpenBazaar GUID to your Blockstack ID, please follow [these instructions](https://github.com/blockstack/blockstack-cli/blob/master/docs/openbazaar.md) instead.
**CAUTION**: You almost never want to update your name's zonefile, since it's slow, tedious, and costs money. It is meant primarily for recovering from zonefile loss, for changing where people find your profile, and for changing your data public key. If you want to store data in your profile, please see the [data storage](https://github.com/blockstack/blockstack-cli/blob/master/docs/advanced_usage.md#data-storage) and [accounts](https://github.com/blockstack/blockstack-cli/blob/master/docs/advanced_usage.md#accounts) commands in the [advanced usage](https://github.com/blockstack/blockstack-cli/blob/master/docs/advanced_usage.md) section (but read the [warnings](https://github.com/blockstack/blockstack-cli/blob/master/docs/advanced_usage.md#a-word-of-warning) first).
If you want to change the name's zonefile, you can do so with `blockstack update`. You must specify the new zonefile to do so. For example:
```
$ blockstack update judecn.id '$ORIGIN judecn.id
> $TTL 3600
> pubkey TXT "pubkey:data:04cabba0b5b9a871dbaa11c044066e281c5feb57243c7d2a452f06a0d708613a46ced59f9f806e601b3353931d1e4a98d7040127f31016311050bedc0d4f1f62ff"
> _file URI 10 1 "file:///home/jude/.blockstack/storage-disk/mutable/judecn.id"
> _https._tcp URI 10 1 "https://blockstack.s3.amazonaws.com/judecn.id"
> _http._tcp URI 10 1 "http://node.blockstack.org:6264/RPC2#judecn.id"
> _dht._udp URI 10 1 "dht+udp://fc4d9c1481a6349fe99f0e3dd7261d67b23dadc5"
> '
```
The zonefile can be any valid DNS zonefile, but must follow these extra rules:
* There must be only one `$ORIGIN`, and it must be the blockstack ID.
* There must be at least one `URI` resource record.
* If you want to set a new data keypair, you must do so via a `TXT` record named `pubkey`, and the text field must start with `pubkey:data:` (as per the example). It must be an ECDSA public key.
**WARNING**: Each of the URLs must refer to the **signed JSON Web Token** (JWT) that encodes your profile data. The JWT can be signed either with the private key that owns your name, or with a private key that matches the `pubkey:data:` TXT record. If you do not do this, your profile **will not be readable** to the Blockstack CLI tool or to any public profile resolvers.

119
docs/glossary.md Normal file
View File

@@ -0,0 +1,119 @@
# Glossary
Commonly used terms and jargon in Blockstack
## Account
A field in a profile that links the name to an existing service, like Twitter or OpenBazaar. They are listed under the `accounts` listing in a profile.
Some accounts serve as social proofs, but they can contain any data the user wants.
## Blockstack ID
(Also called a "name").
A human-readable name in Blockstack. It is comprised only of upper and lower-case ASCII characters, numbers, as well as `-`, `_`, and `.`. It must end with a `.`, followed by a namespace ID. It has at least 3 characters, and at most 37 (including the `.` and the namespace ID).
Anyone can register a Blockstack ID, either via [Onename](https://onename.com) or via the Blockstack CLI. The former is free for most names (subject to spam and squatter filtering), but latter costs Bitcoin.
## Blockstack Server
A server that reads a blockchain with [virtualchain](https://github.com/blockstack/blockstack-virtualchain), filters out transactions that represent name operations, and builds up a database of (name, public key, zonefile hash) triples.
## Consensus Hash
A cryptographic hash that represents a proof-of-computation by a Blockstack Server. Two Blockstack Servers have seen and processed the same name operations up to block `n` if and only if they each calculate the same consensus hash at height `n`.
A Blockstack Server only accepts a name operation if it has a previously-calculated but recent consensus hash. The Blockstack CLI obtains a consensus hash from a Blockstack Server in order to construct a name operation.
## Immutable Data
This is data stored in your storage providers, but linked to by your *zonefile*. It is called "immutable" since its hash is linked to by the blockchain directly. While this is an extremely secure way of sharing data, the downside is that it's slow--each time you change a piece of immutable data, you have to send a new transaction and wait for it to confirm.
Storing data as immutable data is useful when the data is read-only, or changes very rarely.
## Mutable Data
This is data stored in your storage providers, but linked to by your *profile*. It is called "mutable" since it can be changed without touching the blockchain. Mutable data is signed by your wallet's data key, so anyone who reads it can verify that it came from you (i.e. the storage providers aren't trusted with authenticity).
Storing data as mutable data is useful when the data has to change quickly, and you don't really care if other readers won't see some of the changes. This is the recommended way to associating external data with your Blockstack ID. The only downside is that someone who's never read your new data can be tricked into reading old data by a malicious storage provider. If this is a concern, you should use immutable data.
## Name
See Blockstack ID.
## Name Database
The set of (name, public key, zonefile hash) triples that the Blockstack Server generates by reading the blockchain.
## Name Operation
A specially-crafted transaction in the underlying blockchain that, when processed, will change each Blockstack Server's name database. Examples include `NAME_PREORDER` (preorders a name), `NAME_REGISTER` (registers a name), `NAME_UPDATE` (changes a name's zonefile hash), `NAME_TRANSFER` (changes a name's public key), and `NAME_REVOKE` (locks everyone out of a name until it expires).
Name operations are encoded on Bitcoin as `OP_RETURN` outputs that start with `id`, followed by a 1-byte character that identifies the particular operation.
## Namespace
Analogous to a DNS TLD, it represents a grouping of names. All names under the same namespace have the same pricing and lifetime rules.
Anyone can create a namespace, but doing so is expensive by design.
## Preorder
The first of two steps to acquire a name. This operation writes the hash of both the name and the address that will own it.
## Profile
A signed JSON web token that describes a [Person](https://schema.org/Person), which describes the name's owner. You can put anything you want into your profile.
Additionally, profiles hold lists of accounts, and pointers to mutable data.
## Register
(1) The act of acquiring a name in Blockstack.
(2) The second of two steps to create a new name entry in the name database. Reveals the name as plaintext to the world. Must match a recent preorder to be accepted.
## Registrar
An online service that lets you sign up for and manage the profiles of Blockstack IDs. [Onename](https://onename.com) offers a registrar.
## Resolver
An online service that displays zonefile and profile data for a Blockstack ID. [Onename](https://onename.com) offers a resolver.
## Social proof
A post in an account on an existing Web service like Twitter, Facebook, or GitHub that points back to a Blockstack ID. Used to provide some evidence that the person who owns the Blockstack ID is also the person who owns the Web service account.
Social proofs are listed as profile accounts.
## Storage Provider
This is any service that can serve your zonefile, profile, or data. In all cases, the data is signed by one of your wallet's keys, so you can use any provider without having to worry about it trying to change the data.
Not all storage providers support writes--some of them are read-only.
Supported storage providers today include:
* Amazon S3
* Your harddrive
* Onename's Kademila DHT
* Any HTTP/HTTPS/FTP server (read-only)
* Onename's profile resolver (read-only)
* Any Blockstack Server
Support is being added for:
* Dropbox
* Google Drive
* Microsoft OneDrive
* Box.com
* BitTorrent
* IPFS
If you have a preferred storage provider, and you're a developer, please consider sending us a pull request to add support for it!
## Zonefile
A type of file usually used by DNS servers, but used by Blockstack to point the CLI to various storage providers that hold copies of a user's profile information. Its hash is written to the blockchain, and Blockstack Servers constantly try to replicate each zonefile to each other server, so each one has a full replica of all zonefiles in existence.
In addition, zonefiles store your data public key, as well as any pointers to immutable data.

64
docs/openbazaar.md Normal file
View File

@@ -0,0 +1,64 @@
# Linking your OpenBazaar GUID to your Blockstack ID
If you don't have the Blockstack CLI. Download and install it first. Instructions are [here](https://github.com/blockstack/blockstack-cli/blob/master/README.md). The rest of this tutorial assumes that you've already registered a name using the Blockstack CLI.
## Step 1: Advanced Mode
The first step is to activate "advanced mode" in the CLI. The command to do so is:
```
$ blockstack set_advanced_mode on
```
## Step 2: Add an OpenBazaar Account
The second step is to create an OpenBazaar account for your profile (the [Onename](https://onename.com) app also enabled to link your OpenBazaar GUID). The command to do so is:
```
$ blockstack put_account "<BLOCKSTACK ID>" "openbazaar" "<YOUR OB GUID>" "<URL TO YOUR STORE>"
```
The URL can be any valid URL; it won't be used by OpenBazaar. Here's an example, using the name `testregistration001.id` and the GUID `0123456789abcdef`:
```
$ blockstack put_account "testregistration001.id" "openbazaar" "0123456789abcdef" "https://bazaarbay.org/@testregistration001"
```
The update should be instantaneous. You can verify that your store is present with `list_accounts`:
```
$ blockstack list_accounts "testregistration001.id"
{
"accounts": [
{
"contentUrl": "https://bazaarbay.org/@testregistration001.id",
"identifier": "0123456789abcdef",
"service": "openbazaar"
}
]
}
````
# Troubleshooting
Common problems you might encounter.
## Profile is in legacy format
If you registered your blockstack ID before spring 2016, there's a chance that your profile is still in a legacy format. It will get migrated to the new format automatically if you update your profile on the [Onename](https://onename.com) app. However, you have to do this manually with the CLI.
To do so, the command is:
```
$ blockstack migrate <YOUR BLOCKSTACK ID>
```
It will take a little over an hour to complete, but once finished, you'll be able to manage your accounts with the above commands (and do so with no delays).
## Failed to broadcast update transaction
This can happen during a `migrate` for one of a few reasons:
* You do not have enough balance to pay the transaction fee (which is calculated dynamically).
* Your payment address has unconfirmed transactions.
* You can't connect to a Bitcoin node.
To determine what's going on, you should try the command again by typing `BLOCKSTACK_DEBUG=1 blockstack ...` instead of `blockstack...`.

201
pkg/brew/blockstack.rb Normal file
View File

@@ -0,0 +1,201 @@
# generated in part from 'homebrew-pypi-poet'
# to generate:
# * create a virtualenv
# * pip install homebrew-pypi-poet
# * poet -f > /path/to/brewfile
class Blockstack < Formula
desc "Blockstack command-line client"
homepage "https://blockstack.org"
url "https://files.pythonhosted.org/packages/6e/f1/5518019aef7be6cb8726d1aeead954234c1afc2d6c1e2bb39725bb14ca3d/blockstack-0.0.13.6.tar.gz"
sha256 "a665f150ed50bf51874e9f630da44ad8f12c72534c2967eb4fd51f844421b907"
# NOTE: must be added manually, after using `poet`
depends_on :python if MacOS.version <= :snow_leopard
depends_on "openssl"
depends_on "libffi"
resource "base58" do
url "https://files.pythonhosted.org/packages/32/8c/9b8b1b8364a945fa1ed4308d650880a5eb77bd08c2086e32e1f608440ed8/base58-0.2.3.tar.gz"
sha256 "a691b5d194617a3de401aa2ed8818f12f1e348e95524f74a9c67246b59368fff"
end
resource "basicrpc" do
url "https://files.pythonhosted.org/packages/ec/47/516aa9c118a437e833e604d3a35503cf55edcc991aba02d8505811976a57/basicrpc-0.0.2.tar.gz"
sha256 "d471cd0a1f06766583c4b7d6cbae3ab9e2afeb64794b1cee94906cfb4da2a2af"
end
resource "bitcoin" do
url "https://files.pythonhosted.org/packages/12/88/c9390638d5b2713d38ccea46c93e8ec084f052a15a94f9b1d4c66baabd24/bitcoin-1.1.42.tar.gz"
sha256 "11ba70bd9e1c764f6bb2c4bd4c7fbebd5c9053c73f4d4325b00a98869a8b7236"
end
resource "bitmerchant" do
url "https://files.pythonhosted.org/packages/a0/3c/3cc4b1f447cf0ea27c21a0e2e55adaf0064a8cd4b3294fb3c27ca27f6dd3/bitmerchant-0.1.8.tar.gz"
sha256 "fba4c2091084ed0b4f9faac0ebbba8d255c5ccd3a18e7f877ce13aab71649ddc"
end
resource "blockstack-profiles" do
url "https://files.pythonhosted.org/packages/29/13/ff709ac4a0ca5622aee4ee659eb3b214489303f10e8d065745f828510379/blockstack-profiles-0.4.4.tar.gz"
sha256 "e18847f25bf2d264a293e1d0b5304d794e06bea92e947ca7720b08d58927883a"
end
resource "blockstack-storage-drivers" do
url "https://files.pythonhosted.org/packages/12/0c/c21b3159440ca24fd7ec61c4d45776a07b32ce3aab649221f141af756a37/blockstack-storage-drivers-0.0.13.6.tar.gz"
sha256 "ce52d795833ebb22d757734dbd76fec4b329841564c71257119436be03035df0"
end
resource "blockstack-utxo" do
url "https://files.pythonhosted.org/packages/0d/b7/3a86cd1c703c4a12b09d290e5b25a566a20a34261d4aaf01134b051e8a6d/blockstack-utxo-0.0.13.0.tar.gz"
sha256 "5c4728c3b4c450ebc00a681f570a81dfca84061d798da28594e3a4e716a4dcef"
end
resource "blockstack-zones" do
url "https://files.pythonhosted.org/packages/74/8a/db5a5da42d58631568204055c90f9fc2e1790290357c4c18db369a1eb76e/blockstack-zones-0.1.6.tar.gz"
sha256 "d2c946151e149f101e482b1afc0c8bb5d59a397a2236b7811b6d99b5e023debe"
end
resource "boto" do
url "https://files.pythonhosted.org/packages/e9/74/7ef3431c37fc1f51f98cc04491cdb112dcd9f474c83b275e1a1450c24527/boto-2.41.0.tar.gz"
sha256 "c638acdecb0a2383b517c15ac2a6ccf15a2f806aee923cc4448a59b9b73b52e0"
end
resource "cachetools" do
url "https://files.pythonhosted.org/packages/ba/00/b0ec69e21142cd838b2383a7881cf18368e35847cb66f908c8f25bcbaafc/cachetools-1.1.6.tar.gz"
sha256 "d1a44ffd2eedd138f3ba69038feb807ea54cb24e8a207a52d3a8603bc4961821"
end
resource "cffi" do
url "https://files.pythonhosted.org/packages/83/3c/00b553fd05ae32f27b3637f705c413c4ce71290aa9b4c4764df694e906d9/cffi-1.7.0.tar.gz"
sha256 "6ed5dd6afd8361f34819c68aaebf9e8fc12b5a5893f91f50c9e50c8886bb60df"
end
resource "commontools" do
url "https://files.pythonhosted.org/packages/f9/f1/4ef3bdc0ee16d8cd578217f77f39f22a1cd9cb78f469d1ea8efe32b75f4b/commontools-0.1.0.tar.gz"
sha256 "945d510c3950a693d6f7eb9328b18d3efc02bc38d1a89f28f4e9b7411fe9b0cb"
end
resource "cryptography" do
url "https://files.pythonhosted.org/packages/a9/5b/a383b3a778609fe8177bd51307b5ebeee369b353550675353f46cb99c6f0/cryptography-1.4.tar.gz"
sha256 "bb149540ed90c4b2171bf694fe6991d6331bc149ae623c8ff419324f4222d128"
end
resource "defusedxml" do
url "https://files.pythonhosted.org/packages/09/3b/b1afa9649f48517d027e99413fec54f387f648c90156b3cf6451c8cd45f9/defusedxml-0.4.1.tar.gz"
sha256 "cd551d5a518b745407635bb85116eb813818ecaf182e773c35b36239fc3f2478"
end
resource "ecdsa" do
url "https://files.pythonhosted.org/packages/f9/e5/99ebb176e47f150ac115ffeda5fedb6a3dbb3c00c74a59fd84ddf12f5857/ecdsa-0.13.tar.gz"
sha256 "64cf1ee26d1cde3c73c6d7d107f835fed7c6a2904aef9eac223d57ad800c43fa"
end
resource "enum34" do
url "https://files.pythonhosted.org/packages/bf/3e/31d502c25302814a7c2f1d3959d2a3b3f78e509002ba91aea64993936876/enum34-1.1.6.tar.gz"
sha256 "8ad8c4783bf61ded74527bffb48ed9b54166685e4230386a9ed9b1279e2df5b1"
end
resource "idna" do
url "https://files.pythonhosted.org/packages/fb/84/8c27516fbaa8147acd2e431086b473c453c428e24e8fb99a1d89ce381851/idna-2.1.tar.gz"
sha256 "ed36f281aebf3cd0797f163bb165d84c31507cedd15928b095b1675e2d04c676"
end
resource "ipaddress" do
url "https://files.pythonhosted.org/packages/cd/c5/bd44885274379121507870d4abfe7ba908326cf7bfd50a48d9d6ae091c0d/ipaddress-1.0.16.tar.gz"
sha256 "5a3182b322a706525c46282ca6f064d27a02cffbd449f9f47416f1dc96aa71b0"
end
resource "jsontokens" do
url "https://files.pythonhosted.org/packages/71/fe/27f5fc1f61f881b250e65a642eb62f0ac0daddb51fddd3ecb1c85e5f0a64/jsontokens-0.0.2.tar.gz"
sha256 "c63c3a1ddb581a4696fec28eaddb755648144058d8fb369f72db8305de204e46"
end
resource "keychain" do
url "https://files.pythonhosted.org/packages/a8/25/afa689ea2ca8254b52b0d3e5b4f35736bea6cba105d5c394eecf380b50de/keychain-0.1.4.1.tar.gz"
sha256 "e3087ec79f46d8fb618c0d2dda7e474aaf36d98b08dcc3c2375ca8db95227bcd"
end
resource "keylib" do
url "https://files.pythonhosted.org/packages/e0/8e/aac128d5facaac09109f147aaf02d77a6a0617a7724456264d9250723990/keylib-0.0.5.tar.gz"
sha256 "e52878439862f8da9add6547d3fb81f7f9ecf0bb3012799738b98194dbe8862e"
end
resource "mixpanel" do
url "https://files.pythonhosted.org/packages/3b/32/ab8eae3015cb3cb1285d128854357c579e4f6c6e5df174704f750f258e7a/mixpanel-4.3.1.tar.gz"
sha256 "5647dc18ef2a34daae56bc8838bf1f72d0be184fc4bdc0dba6e4c8d00519aa22"
end
resource "protocoin" do
url "https://files.pythonhosted.org/packages/74/ac/18e6c67061166c42cf16381894b6b4f47a993b979fbeaff2f537d723dd55/protocoin-0.2.tar.gz"
sha256 "616aeeb7a7f63a5f4a066cd4019788a4a327d80bd6fc59ce4a3c84222733eb0f"
end
resource "pyasn1" do
url "https://files.pythonhosted.org/packages/f7/83/377e3dd2e95f9020dbd0dfd3c47aaa7deebe3c68d3857a4e51917146ae8b/pyasn1-0.1.9.tar.gz"
sha256 "853cacd96d1f701ddd67aa03ecc05f51890135b7262e922710112f12a2ed2a7f"
end
resource "pybitcoin" do
url "https://files.pythonhosted.org/packages/67/78/d538af65c51032f5b06377baa9713b61e798fe04b96dce6386ca6391a02f/pybitcoin-0.9.9.tar.gz"
sha256 "0348d65232d5299d8619f2e9a0b993a72b2ee3492a71612f61d490b754421371"
end
resource "pycparser" do
url "https://files.pythonhosted.org/packages/6d/31/666614af3db0acf377876d48688c5d334b6e493b96d21aa7d332169bee50/pycparser-2.14.tar.gz"
sha256 "7959b4a74abdc27b312fed1c21e6caf9309ce0b29ea86b591fd2e99ecdf27f73"
end
resource "pycrypto" do
url "https://files.pythonhosted.org/packages/60/db/645aa9af249f059cc3a368b118de33889219e0362141e75d4eaf6f80f163/pycrypto-2.6.1.tar.gz"
sha256 "f2ce1e989b272cfcb677616763e0a2e7ec659effa67a88aa92b3a65528f60a3c"
end
resource "python-bitcoinrpc" do
url "https://files.pythonhosted.org/packages/2b/bd/54ce6bb445330df0938a91bc2bcac472142a5c57e3e0b329958992a352c4/python-bitcoinrpc-0.1.tar.gz"
sha256 "6306ab38bb73d7399f2a037c53e2f63f4445ba336f85a8e4055f005f3bf3a97f"
end
resource "requests" do
url "https://files.pythonhosted.org/packages/49/6f/183063f01aae1e025cf0130772b55848750a2f3a89bfa11b385b35d7329d/requests-2.10.0.tar.gz"
sha256 "63f1815788157130cee16a933b2ee184038e975f0017306d723ac326b5525b54"
end
resource "six" do
url "https://files.pythonhosted.org/packages/b3/b2/238e2590826bfdd113244a40d9d3eb26918bd798fc187e2360a8367068db/six-1.10.0.tar.gz"
sha256 "105f8d68616f8248e24bf0e9372ef04d3cc10104f1980f54d57b2ce73a5ad56a"
end
resource "utilitybelt" do
url "https://files.pythonhosted.org/packages/ab/31/343ef1df18ffe822f02b4ca879d1f406275d3187040ac724bcf9158e4669/utilitybelt-0.2.6.tar.gz"
sha256 "dafdb6a2dbb32e71d67a9cd35afd7c2e4993ec094e7ddb547df4cf46788770a4"
end
resource "virtualchain" do
url "https://files.pythonhosted.org/packages/19/31/4130ce07b22d43e8b9412459175ff949be037a6c9d64236bee7a00a013bb/virtualchain-0.0.13.2.tar.gz"
sha256 "bf74c4d5ff9d3d6509fc0c222e15915ca34b4d0a8fde393af2c0c8e04e1268ac"
end
def install
ENV.prepend_create_path "PYTHONPATH", libexec/"vendor/lib/python2.7/site-packages"
# NOTE: this extra path must be added manually, in order for cryptography to work (since it uses pkg_resources, which must be the one *brew* installed, not the OS X one (which is out of date))
ENV.prepend_create_path "PYTHONPATH", "/usr/local/lib/python2.7/site-packages"
%w[base58 basicrpc bitcoin bitmerchant blockstack-profiles blockstack-storage-drivers blockstack-utxo blockstack-zones boto cachetools cffi commontools cryptography defusedxml ecdsa enum34 idna ipaddress jsontokens keychain keylib mixpanel protocoin pyasn1 pybitcoin pycparser pycrypto python-bitcoinrpc requests six utilitybelt virtualchain].each do |r|
resource(r).stage do
system "python", *Language::Python.setup_install_args(libexec/"vendor")
end
end
ENV.prepend_create_path "PYTHONPATH", libexec/"lib/python2.7/site-packages"
system "python", *Language::Python.setup_install_args(libexec)
bin.install Dir[libexec/"bin/*"]
bin.env_script_all_files(libexec/"bin", :PYTHONPATH => ENV["PYTHONPATH"])
end
test do
false
end
end

619
tools/namespace_import.py Executable file
View File

@@ -0,0 +1,619 @@
#!/usr/bin/python
"""
Blockstack-client
~~~~~
copyright: (c) 2014-2015 by Halfmoon Labs, Inc.
copyright: (c) 2016 by Blockstack.org
This file is part of Blockstack-client.
Blockstack-client is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
Blockstack-client is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with Blockstack-client. If not, see <http://www.gnu.org/licenses/>.
"""
import random
import json
import time
import os
import sys
import traceback
import subprocess
import pprint
import pybitcoin
import binascii
import logging
import requests
import keychain
DEBUG = True
from ConfigParser import SafeConfigParser
# Hack around absolute paths
current_dir = os.path.abspath(os.path.dirname(__file__))
parent_dir = os.path.abspath(current_dir + "/../")
sys.path.insert(0, parent_dir)
AVG_BLOCK_TIME = 600 # average number of seconds between blocks
CONFIRM_DELAY = 10 # number of blocks to wait to pass before confirming that the name was registered
MAX_UNCONFIRMED = 50000000
def pretty_dump(json_str):
""" pretty dump
"""
return json.dumps(json_str, sort_keys=True, indent=4, separators=(',', ': '))
def send_checkpoint( chaincom_client, privkey_str, checkpoint_str="http://blockstack.org/summit" ):
"""
Write an OP_RETURN to the blockchain, to checkpoint our progress.
Should be written every Nth name.
"""
nulldata = binascii.hexlify( checkpoint_str )
return pybitcoin.embed_data_in_blockchain( nulldata, privkey_str, chaincom_client, 10000, format='hex')
def namecoin_to_bitcoin_address( nmc_address ):
"""
Convert a namecoin address to a bitcoin address.
The only difference is the version number.
"""
return pybitcoin.b58check_encode( pybitcoin.b58check_decode( nmc_address ), version_byte=0 )
def get_chaincom_api_keys( path="./chaincom.ini" ):
"""
Read the ini-formatted file given to find chain.com keys.
"""
parser = SafeConfigParser()
parser.read( path )
return parser.get("chaincom", "api_key_id"), parser.get("chaincom", "api_key_secret")
def get_num_unconfirmed_txs( chaincom_api_key, chaincom_api_secret, address, count=500 ):
"""
Get the number of unconfirmed transactions for an address.
"""
r = requests.get("https://api.chain.com/v2/bitcoin/addresses/%s/transactions?limit=%s&api-key-id=%s&api-key-secret=%s" % (address, count, chaincom_api_key, chaincom_api_secret))
data = None
if r.status_code != 200:
return -1
try:
data = r.json()
except:
print >> sys.stderr, "Failed to get transactions"
return None
num_unconfirmed = 0
for tx in data:
try:
if tx["confirmations"] == 0:
num_unconfirmed += 1
except:
print tx
sys.exit(1)
return num_unconfirmed
def get_balance( chaincom_api_key, chaincom_api_secret, address ):
"""
Get the number of unconfirmed transactions for an address.
"""
r = requests.get("https://api.chain.com/v2/bitcoin/addresses/%s?api-key-id=%s&api-key-secret=%s" % (address, chaincom_api_key, chaincom_api_secret))
data = None
if r.status_code != 200:
return -1
try:
data = r.json()
except:
print >> sys.stderr, "Failed to get transactions"
return None
return data[0]["total"]["balance"]
def confirm_name_imported( client, name ):
"""
See if a name has been imported.
"""
name_info = client.lookup( name )
if 'error' in name_info:
log.info( "confirm '%s'...no" % name )
return False
# must be a full record
name_info = name_info[0]
if name_info is None:
log.info( "confirm '%s'...no" % name )
return False
if 'address' not in name_info:
log.info( "confirm '%s'...no" % name )
return False
if 'value_hash' not in name_info:
log.info( "confirm '%s'...no" % name )
return False
if name_info['value_hash'] is None:
log.info( "confirm '%s'...no" % name )
return False
log.info( "confirm '%s'...yes" % name )
return True
def find_imported( client, name_list ):
"""
Find the list of names that have been imported
"""
imported = []
for name in name_list:
if confirm_name_imported( client, name ):
imported.append( name )
return imported
if __name__ == "__main__":
names_json = None
names = None
namespace_id = None
privkey_str = None
pp = pprint.PrettyPrinter()
if len(sys.argv) != 4:
print >> sys.stderr, "Usage: %s [json_file] [namespace_id] [private_key]"
sys.exit(1)
# try to connect to our broadcast provider
try:
chaincom_id, chaincom_secret = get_chaincom_api_keys()
except Exception, e:
traceback.print_exc()
print >> sys.stderr, "Failed to get Chain.com API keys"
sys.exit(1)
chaincom_client = pybitcoin.ChainComClient( chaincom_id, chaincom_secret )
# get our namespace's names
try:
with open( sys.argv[1], "r" ) as f:
names_json = f.read()
except Exception, e:
traceback.print_exc()
print >> sys.stderr, "Failed to read '%s'" % sys.argv[1]
sys.exit(1)
privkey_str = sys.argv[3]
namespace_id = sys.argv[2]
print "--------------------------------------------------------------"
print "WARN: you will need to populate these keys with BTC beforehand"
print "--------------------------------------------------------------"
total_balance = 0
keyring = []
unfunded = []
keyring_path = "%s.keyring" % namespace_id
if os.path.exists(keyring_path):
print "import from '%s'" % keyring_path
try:
tmp = []
with open( keyring_path, "r" ) as f:
tmp = f.readlines()
keyring = []
tmp2 = [k.strip() for k in tmp]
for pk in tmp2:
addr = pybitcoin.BitcoinPrivateKey( pk ).public_key().address()
balance = get_balance( chaincom_id, chaincom_secret, addr )
print "%s (%s) balance: %s" % (pk, addr, balance)
total_balance += balance
if balance > 54000:
keyring.append( pk )
else:
unfunded.append(addr)
except Exception, e:
log.exception(e)
pass
if len(keyring) == 0:
pk = pybitcoin.BitcoinPrivateKey( privkey_str )
keyring_generator = keychain.PrivateKeychain.from_private_key( privkey_str )
keyring = [ pk.to_hex() ]
print "%s (%s) master" % (pk.to_wif(), pk.public_key().address())
for i in xrange(0, 300):
pk_hex = keyring_generator.child(i).private_key()
pk_wif = pybitcoin.BitcoinPrivateKey( pk_hex ).to_wif()
pk_addr = pybitcoin.BitcoinPrivateKey( pk_hex ).public_key().address()
balance = get_balance( chaincom_id, chaincom_secret, pk_addr )
print "%s (%s) balance: %s" % (pk_wif, pk_addr, balance)
total_balance += balance
if balance >= 54000:
keyring.append( pk_wif )
else:
unfunded.append(pk_addr)
try:
with open(keyring_path, "w+") as f:
for k in keyring:
f.write("%s\n" % k)
f.flush()
except Exception, e:
log.exception(e)
pass
print "--------------------------------------------------------------"
print "Unfunded:"
for addr in unfunded:
print addr
print "--------------------------------------------------------------"
print "Total: %s" % total_balance
print "--------------------------------------------------------------"
try:
names = json.loads( names_json )
except Exception, e:
traceback.print_exc()
print >> sys.stderr, "Invalid JSON file '%s'" % sys.argv[1]
sys.exit(1)
# record name status
logfile_path = namespace_id + ".sent"
failed_path = namespace_id + ".failed"
confirmed_path = namespace_id + ".confirmed"
sent_names_json = []
sent_names = {}
confirmed_names = []
unconfirmed_names = {} # map name to send time
failed_lines = None
failed = []
num_sent_names = 0
# resume from where we left off...
try:
# should contain newline-separated list of names we've processed so far
if not os.path.exists( logfile_path ):
sent_fd = open( logfile_path, "w+" )
else:
sent_fd = open( logfile_path, "r+" )
if not os.path.exists( failed_path ):
failed_fd = open( failed_path, "w+" )
else:
failed_fd = open( failed_path, "r+" )
if not os.path.exists( confirmed_path ):
confirmed_fd = open( confirmed_path, "w+" )
else:
confirmed_fd = open( confirmed_path, "r+" )
sent_names_json = sent_fd.read().split("\n")
confirmed_names = confirmed_fd.read().split("\n")
failed = failed_fd.read().split("\n")
except Exception, e:
traceback.print_exc()
print >> sys.stderr, "Failed to open '%s'" % logfile_path
sys.exit(1)
# get list of sent name records
for name_json in sent_names_json:
if len(name_json.strip()) == 0:
continue
name_rec = json.loads( name_json.strip() )
sent_names[ name_rec['name'] ] = name_rec
# connect to blockstack
conf = config.get_config()
proxy = client.session(conf['blockstackd_server'], conf['blockstackd_port'])
time_of_last_confirmation = 0
# find any previously-sent-but-not-confirmed names
for sent_name in sent_names.keys():
if sent_name not in confirmed_names:
t = sent_names[sent_name].get('time', 0)
unconfirmed_names[ sent_name ] = t
# sanity check all names
for name in names:
# must have BTC or NMC address
if not name.has_key('nmc_address') and not name.has_key('btc_address'):
raise Exception("Name '%s' lacks an address" % name)
# must have a profile hash
if not name.has_key('hash') and not name.has_key('profile_hash'):
raise Exception("Name '%s' lacks a profile hash" % name )
# some failed names might have actually gone through. Find out which.
new_confirmed = find_imported( client, failed )
for confirmed_name in new_confirmed:
confirmed_fd.write( "%s\n" % confirmed_name )
confirmed_fd.flush()
if confirmed_name in sent_names.keys():
del sent_names[confirmed_name]
confirmed_names.append( confirmed_name )
failed.remove( confirmed_name )
# new failed set
new_failed_fd = open( failed_path + ".tmp", "w+" )
new_failed_fd.write( "\n".join( failed ) )
new_failed_fd.flush()
new_failed_fd.close()
os.rename( failed_path + ".tmp", failed_path )
key_rr = 0
# do all imports
for name in names:
username = name['username']
if name.has_key('hash'):
update_hash = str(name['hash'])
elif name.has_key('profile_hash'):
update_hash = str(name['profile_hash'])
fqn = username + "." + namespace_id
if fqn in confirmed_names or fqn in sent_names.keys():
# already imported or sent
num_sent_names += 1
continue
# how long are we doing to wait?
delay = 20
try:
with open("delay.txt", "r") as fd:
delay_txt = fd.read()
delay = float(delay_txt.strip())
except Exception, e:
print >> sys.stderr, "failed to read delay.txt; assuming %s" % delay
"""
# every so often, see if we need to throttle ourselves
if num_sent_names % len(keyring) == 0:
total_unconfirmed = 0
num_unconfirmed_txs = 10000000000
while num_unconfirmed_txs > MAX_UNCONFIRMED:
total_unconfirmed = 0
for pk_str in keyring:
addr = pybitcoin.BitcoinPrivateKey( pk_str ).public_key().address()
total_unconfirmed += get_num_unconfirmed_txs( chaincom_id, chaincom_secret, addr )
num_unconfirmed_txs = total_unconfirmed
print >> sys.stderr, "%s unconfirmed transactions" % num_unconfirmed_txs
if num_unconfirmed_txs <= MAX_UNCONFIRMED:
break
time.sleep(60)
"""
# every block (or on start-up), update the list of imported names
if time_of_last_confirmation + AVG_BLOCK_TIME < time.time():
log.info( "Check for confirmed names" )
# get sent names that were sent more than AVG_BLOCK_TIME * CONFIRM_DELAY seconds ago
names_to_check = filter( lambda n: unconfirmed_names[n] + AVG_BLOCK_TIME * CONFIRM_DELAY < time.time(), unconfirmed_names.keys() )
# which of our unconfirmed names have been confirmed?
new_confirmed = find_imported( client, names_to_check )
for confirmed_name in new_confirmed:
confirmed_fd.write( "%s\n" % confirmed_name )
confirmed_fd.flush()
if confirmed_name in sent_names.keys():
del sent_names[confirmed_name]
confirmed_names.append( confirmed_name )
del unconfirmed_names[ confirmed_name ]
time_of_last_confirmation = time.time()
btc_address = None
nmc_address = None
address = None
if name.has_key('nmc_address'):
nmc_address = str(name['nmc_address'])
btc_address = namecoin_to_bitcoin_address( nmc_address )
elif name.has_key("btc_address"):
btc_address = name['btc_address']
# ascii only
try:
fqn = str(fqn)
except:
log.debug("non-ascii name '%s'" % fqn)
continue
count = 0
MAX_COUNT = len(keyring)
already_exists = False
while count < MAX_COUNT:
pk_str = keyring[ key_rr % len(keyring) ]
pub_str = pybitcoin.BitcoinPrivateKey( pk_str ).public_key().address()
key_rr += 1
existing_name = client.lookup( fqn )
try:
if existing_name[0] is not None:
log.debug("Name '%s' already imported: %s" % (fqn, existing_name))
confirmed_fd.write( "%s\n" % fqn )
confirmed_fd.flush()
confirmed_names.append( fqn )
already_exists = True
break
except Exception, e:
log.exception(e)
sys.exit(0)
log.debug( "name_import " + fqn + " " + btc_address + " " + update_hash + " (key " + pub_str + ")" )
try:
result = client.name_import( fqn, btc_address, update_hash, pk_str )
except Exception, e:
log.error( "register '%s' failed:\n%s\n" % (fqn, traceback.format_exc()) )
if count == 0:
failed_fd.write( "%s\n" % (fqn))
failed_fd.flush()
count += 1
continue
if type(result) == type([]):
result = result[0]
if 'error' in result.keys():
log.error( "register '%s' failed:\n%s\n" % (fqn, pp.pformat(result)) )
if result['error'] == 'Name already registered':
# it's because this is confirmed already!
log.debug("Confirmed: '%s'" % fqn)
confirmed_fd.write( "%s\n" % fqn)
confirmed_fd.flush()
if fqn in sent_names.keys():
del sent_names[fqn]
confirmed_names.append( fqn )
if fqn in unconfirmed_names[ fqn ]:
del unconfirmed_names[ fqn ]
elif 'has non-base-38 characters' in result['error']:
break
else:
print >> sys.stderr, "import failed; retry count %s" % count
if count == 0:
failed_fd.write( "%s\n" % (fqn))
failed_fd.flush()
if count == MAX_COUNT - 1:
# try again later
names.append( name )
log.error("result: %s" % result)
count += 1
continue
else:
result['name'] = fqn
result['time'] = time.time()
result_str = json.dumps( result )
sent_fd.write( "%s\n" % result_str)
sent_fd.flush()
print pretty_dump( result )
unconfirmed_names[ fqn ] = result['time']
# record progress
num_sent_names += 1
break
if not already_exists:
time.sleep(delay)
"""
if (num_sent_names % 20 == 0):
checkpoint_tx = send_checkpoint( chaincom_client, privkey_str )
print pretty_dump( checkpoint_tx )
time.sleep(20)
"""
# wait for all names to confirm
while len(unconfirmed_names.keys()) > 0:
time.sleep( AVG_BLOCK_TIME )
# get sent names that were sent more than AVG_BLOCK_TIME * CONFIRM_DELAY seconds ago
names_to_check = filter( lambda n: unconfirmed_names[n] + AVG_BLOCK_TIME * CONFIRM_DELAY < time.time(), unconfirmed_names.keys() )
log.info( "Check for %d unconfirmed name(s)" % len(names_to_check) )
# which of our unconfirmed names have been confirmed?
new_confirmed = find_imported( client, names_to_check )
for confirmed_name in new_confirmed:
confirmed_fd.write( "%s\n" % confirmed_name )
confirmed_fd.flush()
if confirmed_name in sent_names.keys():
del sent_names[confirmed_name]
confirmed_names.append( confirmed_name )
del unconfirmed_names[ confirmed_name ]