This is a story about collaborating on the Open Food Chain, an open-source blockchain integration that began on the Notary Network of Komodo. The Notary Nodes checkpoint the Komodo blockchain and all Komodo Smart Chains into bitcoin's ledger. After 2 seasons, the Komodo Pioneers v1 had split into season 5 Komodo Pioneers v2 & Mylo. However, there was a lot of friendly collaboration on funding meetups in pre-covid times. Bart from the pioneers would host blockchain enthusiasts in The Netherlands. Mylo would look after successful Komodo Pioneers nodes.

Six months after splitting, Bart asked Mylo if he'd be interested in a "proof of concept 3-month project or something". Mylo was quite excited about working on a non-fintech application of the technology we all built at Komodo stemming from the brilliant work of jl777.

Partners

The New Fork was already in the blockchain game, albeit with Hyperledger consulting - but what they really needed for the food & agriculture industries was a true p2p offering of their own, where supply chain partners could rely on the technology without needing to trust a third party with shared data. Enter a Komodo Smart Chain!

The stakeholders for this project began with the aggregate of multi-billion dollar companies and associations:
* Refresco, with their press release
* Austria Juice
* Eckes Granini
* IDH, The Sustainable Trade Initiative, with their press release
* Dozens of news aggregation sites like fruit-processing

Refresco JuicyChain sustainability quote

Technology

We were working on Apple Juice.

After 6 months of long hours and hard work, and as at writing, we are within a week of the "soft launch" of the project running on Google Cloud Platform Kubernetes infrastructure. We are also within a week of the 2021 Season 5 Notary Node elections finishing!

Our small team beat IBM at implementing a p2p solution for global agriculture. Parts of the solution include:
* Running full nodes
* Running explorers
* Using KV storage for decentralized address book
* Managing offline wallets
* Signing offline transactions with electrumx client library
* No transaction fees

These tech solutions will be expanded upon further using the open food chain source code throughout our collaboration & beyond. On behalf of Bart from the Komodo Pioneers and Mylo from iKomodo.com & KomoDeFi.com, we'd like to thank the Komodo user & development community for enabling us to begin collaboration with meetups. This effort would not have been possible without the innovation of checkpoint blockchain security with the Notary Nodes and with the support of community voting to enable us to work on our Komodo co-creations.

Voting

Mylo (Southern Hemisphere) RARdqVRm3RWvii2kCWZBSteqqZQs7PsVj9

AND

Komodo Pioneers (Bart/Bart/Audo) RLDs9ZTtzAQy8Y7eoMht7oMFuciECiqCEi

This post is about the resources required to handle some network users during a live cryptocurrency decentralized exchange test run with genuine users in the wild. The machine operator goes by the discord name cipi & has contributed some monitoring screenshots on network bandwidth, cpu loads & number of connected clients to the SPV infrastructure. To try out some komodo smart chain technology with electrum server, read the tutorial and fire up your own docker image with the blockchain dev kit.

AtomicDEX Stress Test Weekend Event

The stress test event took place on December 12 & 13, 2020. 53 unique addresses achieved more than 1000 swaps each, with the top 5 addresses combining for 90,000 cross chain atomic swaps. The total number of atomic swaps was >200,000 in one weekend.

Dashboard from the https://stresstest.atomicdex.io/dashboard page

SPV Electrum Server Infrastructure

Cipi has stated in discord that the Electrumx server infrastructure used dedicated servers. The clients connected to 3 separate servers. The servers were hosting Bitcoin, Litecoin, Komodo, and the parity Ethereum & ERC20 stuff.

Server Hardware

AMD EPYC 7502P, 384 GB RAM, 3x NVME SSDs in a RAID 0 configuration.

Number of Client Connections

On one of the servers, the maximum number of clients connected at the same time was 444 on the Komodo blockchain.

electrum server connection count

One of the test coins used for distributing to the public is called RICK. This enabled the marketmakers to create an order to sell KMD for these freely distributed RICK coins, thus rewarding testers! The peak maximum number of simultaneous connections on this same server for RICK coin was 398.

electrum server connection count 2

CPU Load

The hardware of each server has 64 threads and 384GB of RAM. They are beasts. The machines ran with minimum load, despite being able to handle approximately 400 concurrent users, sync 5+ blockchains.

electrum server load cpu idle time
Very idle with this event taking place, plenty of room to grow
electrum server load cpu load average
Ignore the yellow line cipi said, that is for his own record keeping

Bandwidth

Bandwidth consumption is often more of a metric to be aware of when hosting multiple blockchains, especially when the number of new full node clients connecting want to sync. Rackspace comes with generally 5TB of data allowance. For lite clients, which this electrum server infrastructure is used mainly for. Looks like a constant 3Mb/s in requests and 6Mb/s in responses is required for hosting 500+ clients including syncing full node chains not used in the stress test.

electrum server load bandwidth

Conclusion

These are ball park stats and no meaningful analysis can be made except the potential to host magnitude more clients is possible with this single server. To host many magnitudes more, simply making geo-DNS queries available to users software and running more servers will work.

Thank you cipi for providing these graphs.

This howto is for using a blockchain dev kit using Komodo, electrumx & atomicDEX. Using 3 existing containers and small mods, you can have HELLOWORLD trading with your team in less than one hour. Improvements to this to totally automate it are in the works to make a turnkey blockchain dev kit.

Our smart chain created and ready to trade on atomicDEX after using the blockchain dev kit
The HELLOWORLD smart chain ready to trade for some MORTY test coins on the atomicDEX network. Why not for BTC or alts!

End To End Blockchain Development Using The Dev Kit

If you are on linux simply download and run the following commmands. It will download 1GB of cryptography stuff.

wget https://raw.githubusercontent.com/KomodoPlatform/komodo/master/zcutil/fetch-params.sh
chmod +x fetch-params.sh
./fetch-params.sh

Download the containers for this particular tutorial. You will need zcash params installed on your machine.

  1. docker pull coingateways/blockchain-starter-kit:5000f2a.3
  2. docker pull coingateways/atomicdexorderbook:0.9.2
  3. docker pull lukechilds/electrumx:v1.15.0

Blockchain Dev Kit

I made a video on the blockchain dev kit in 2019. The current simplified version is similar. Watch from the 2:50 mark for what it does.

Video of the blockchain dev kit in action
Video from 2019 of the blockchain dev kit during testing

To start this container, after having the zcash-params installed type:

docker run -i -t -v /home/${USER}/.zcash-params:/root/.zcash-params:ro --name bdk -p 127.0.0.1:8096:8096 coingateways/blockchain-starter-kit:5000f2a.3

It will drop you into a bash, and you will want to change to the /komodo-in-a-box directory to execute setup/start.sh

cd /komodo-in-a-box ; setup/start.sh

cat ~/.komodo/HELLOWORLD/HELLOWORLD.conf  
rpcuser=user4130001545
rpcpassword=pass26cc5f967a6c9e7c7dd35675429b5f6257f5b7f1a87a7f762bd4fba3553401075e
rpcport=8096
server=1
txindex=1
rpcworkqueue=256
rpcallowip=127.0.0.1
rpcallowip=172.17.0.1
rpcallowip=172.17.0.2
rpcallowip=172.17.0.3
rpcallowip=172.17.0.4
rpcallowip=172.17.0.5
rpcallowip=172.17.0.6

You will need the rpcuser & rpcpassword for the electrumx container!

Start the bdk console again with the command above cd /komodo-in-a-box ; setup/start.sh. And follow the prompts to create a mining node, but don't start mining yet. Let's get the rest of it setup.

Lite Wallet Coin Infrastructure With ElectrumX

This docker container comes from Luke Childs who used to be an Komodo notary node operator and the hyperDEX lead dev. We want to connect electrumx to the seed node. We need to know the IP address of the BDK (blockchain dev kit) container. From your workstation, issue docker inspect bdk | grep -i ipaddress

docker inspect bdk | grep -i ipaddress
           "SecondaryIPAddresses": null,
           "IPAddress": "172.17.0.4",
                   "IPAddress": "172.17.0.4",

I have other containers, running so my BDK for this writing is at 172.17.0.4. I'm using this address to start electrumx with this command. I'm also using the rpcuser & rpcpassword and rpcport from my HELLOWORLD.conf file above.

docker run -e DAEMON_URL=http://user4130001545:pass26cc5f967a6c9e7c7dd35675429b5f6257f5b7f1a87a7f762bd4fba3553401075e@172.17.0.4:8096 -e COIN=Komodo -p 50002:50002 -p 50001:50001 -p 8008:8000 --name electrumx lukechilds/electrumx
Generating a RSA private key
.......................................................+++++
.............+++++
writing new private key to '/data/electrumx.key'
-----
INFO:electrumx:ElectrumX server starting
INFO:electrumx:logging level: INFO
INFO:Controller:Python version: 3.7.7 (default, May 20 2020, 21:33:25)  [GCC 9.2.0]
INFO:Controller:software version: ElectrumX 1.15.0
INFO:Controller:aiorpcX version: 0.18.4
INFO:Controller:supported protocol versions: 1.4-1.4.2
INFO:Controller:event loop policy: None
INFO:Controller:reorg limit is 800 blocks
INFO:Daemon:daemon #1 at 172.17.0.4:8096/ (current)
INFO:DB:switching current directory to /data
INFO:DB:using leveldb for DB backend
INFO:DB:created new database
INFO:DB:creating metadata directory
INFO:DB:UTXO DB version: 8
INFO:DB:coin: Komodo
INFO:DB:network: mainnet

Great! Electrum is up.

AtomicDEX Lite Wallet & P2P Orderbook Trading

The orderbook app for atomicDEX already has a tutorial, but we need to add our HELLOWORLD coin, this requires one command in reality, but for this tutorial let's kill everything the container starts. First we need to know some info.

  1. The IP address of our lite wallet infrastructure so the atomicDEX wallet can connect needs to be known
  2. Tell marketmaker to use HELLOWORLD infrastructure

Again turning to our workstation, we need to issue the same docker inspect command but for the electrumx container.

docker inspect electrumx | grep -i ipaddress
           "SecondaryIPAddresses": null,
           "IPAddress": "172.17.0.5",
                   "IPAddress": "172.17.0.5",

The orderbook container needs starting and it will automatically start everything, we just need to enable HELLOWORLD with curl.

docker run -i -t -p 127.0.0.1:7780:7780 -p 127.0.0.1:8001:8001 -e MMBOT_URL="http://localhost:7780" -e ME_PUBLIC="false" -e ME_PRIVATE="true" --name orderbook coingateways/atomicdexorderbook:0.9.2

Let's enable HELLOWORLD!

source /usr/local/bin/userpass.txt ; curl --url "http://127.0.0.1:7783" --data "{\"userpass\":\"$userpass\",\"method\":\"electrum\",\"coin\":\"HELLOWORLD\",\"servers\":[{\"url\":\"172.17.0.5:50001\"}], \"mm2\": 1}"

Send coins from mining node to browser wallet

Jump back into the container running the blockchain dev kit at the start. You will need to go to the mining node and start mining. Follow the menus for HELLOWORLD mining node. Spin one up, start mining. Go into the wallet menu and import the dev wallet. You will have 1000 coins to send to your lite wallet in the orderbook app.

When Komodo developed the Antara framework, it was based on the agnostic open standard smart contract work from Ripple. Cryptoconditions is a portable multi level, multi algorithm & multi signature capable smart contract format.

The original set of Antara modules are based on the early work of jl777. The vision was to modernize and democratize smart chain building blocks for future product development in the blockchain space. These modules are coded in C++ and leverage the cryptoconditions open standard to enable portability between languages.

New Smart Contract Language Support

In Q1 2020, the Komodo team had already started looking into using other languages for cryptoconditions support (Antara support). These included python and javascript. Some integration tests in the komodo codebase have been in use since 2018. These were originally written in python - so it's an easy bridge to cross using existing implementations of the open standard.

Electrum Lite Wallet Smart Contract Bounty

Although the Antara framework is the most powerful UTXO smart contract implementation in existence, it currently has a high barrier to entry for users. Version 1 was for power users and developers to become familiar with in product development. For wider adoption of the technology one this is necessary. Lite wallets developers need to have access to these powerful functions without requiring the full node operating. This is a limitation for mobile apps, however it is being overcome in two ways. Electrum decoding (python) with a bounty currently available for skilled python protocol developers, and with nSPV technology.

nSPV Super Lightweight Full Node Blockchain

Notarized Simple Payment Verifications allow for a low power computing device to be able to verify & create transactions. Just like a full node, these clients can do it without the need of downloading all the blockchain data. This solution was inspired by the flyclient whitepaper. More details from lead developer jl777 discovery & implementation. The notarization process of dPoW makes this solution possible.

Antara V2 Refactor & Optimization

In 2019 & 2020, through the use of the Antara framework, some early limitations were discovered. The CDP developer from the Komodo team, Mihailo was first to mention the refactor process in discord. The platform sync feature of Komodo to enable cross-chain tokenized asset transfer was first being tested in Q3 2018.

Return Of A Champion + WASM

One of the most impressive developers in Komodo's history has returned to collaborate yet again. Scott Sadler or libscott worked with jl777 in 2017 & 2018 to bring cryptoconditions (& Antara) to the UTXO blockchain world. He has returned and give some clues as to what is coming for Komodo in the mid-term future.

We have cryptoconditions in Rust now also, can be compiled to wasm:

Source of the comment can be found in this issue

Transaction Deconstruction

Many valuable lessons have been learned from the first 18 months of experimenting with what power Antara brings to the UTXO blockchain world. By creating powerful tokenized versions of more valuable and liquid cryptocurrencies, like bitcoin, and exploring the current limitations to advance the science of computing with cryptography the developers previously in jl777's shadow are forging a bright future for sovereign community based projects.

The Antara framework beginning to look like network protocol by encapsulating it's structure in a more elegant and developer friendly envelope.

Portable Smart Contracts

With the introduction of cleaner more standardized internals, and the ability to run wasm cryptoconditions in web apps and many other target computing platforms, integration with other platforms like the wavelets from Perlin (written in rust) or Cosmos' wasm projects becomes more of a picture within the scope of integration rather than a pipe dream. One must not forget that Polkadot is also a Rust implementation of multi-chains, with many DeFi projects already in late stages of development.

The full stack atomic swap tutorial only gets better! Here you will have your own web app for exploring the atomicDEX p2p orderbook market. Market pairs not available on centralized exchanges exist on this p2p trading network. Community exotics such as RVN/DGB directly being listed with DEX prices. No intermediary currency (like BTC/ETH/USDT/BUSB/etc.) and certainly no intermediary party. Your keys your coins!

In part 1 of the tutorial, setting up the telegram bot which relayed our wallet passphrase was helpful. If you did not do it, the passphrase is available in plain text in the root of the container at /passphrase.txt - more advanced docker options to share a file on the host to the container are also available which will be covered at the end.

Viewing Market Orders From A Web App

When I wrote part 1, it was designed for the types of automated testing that can be done without a GUI. Fire up 50 docker instances, with some accompanying scripts. This is ideally what the Komodo dev team would want to use for load testing the network from several data centres around the globe. This image has the web GUI enabled. You need docker installed to play along!

Step 1 - Pulling Updated Tutorial Image

docker pull coingateways/atomicdexorderbook:0.8.4

Step 2 - Start AtomicDEX API, Middleware & Web Front End

For this I'm going to only leave instructions for a telegram bot. If you are not going to use a telegram bot, just omit the (-e) environment variables, it will still work without a problem. Please don't send large amounts of funds without properly having the passphrase backed up. It is auto-generated randomly.

If you have run through tutorial 1, you will want to stop the running container (if you haven't) and remove it. Make sure you have the passphrase saved if you did adventurous things on your own accord (like sending real value to the address). You can recover the funds using the Verus Desktop Wallet.

docker run -i -t -p 127.0.0.1:7780:7780 -p 127.0.0.1:8001:8001 -e MMBOT_URL="http://localhost:7780"  -e BOT_TOKEN="XXXXXXXXX" -e BOT_CHATID="XXXXX" -e BOT_USERNAME="XXX" -e ME_PUBLIC="false" -e ME_PRIVATE="true" --name orderbook coingateways/atomicdexorderbook:0.8.4

We have added the web port to open communications for the browser to connect to on port 8001.

We have also added two variables to pass in ME_PUBLIC & ME_PRIVATE. They are simple true/false values. Best results are reached when they are opposite of each other. ME_PRIVATE enables more dangerous operations from a financial risk point of view e.g. making trade orders broadcast on the network with a funded account.

Step 3 - Browse To Application

Wait! The front end app needs to be compiled - nothing from your end needs to be done, but there will be about a 10 second start up whilst the javascript is optimized. The application is written in VueJS with Material Design baked in from Google. The project VuetifyJS is used for this seamless developer experience.

Browse to http://localhost:8001

The browser will redirect to the dashboard and you will likely see 60+ wallets showing zero balance.

So easy!

Step 4 - Browse P2P Crypto Markets

Currently, atomicDEX is in beta, so there are not many users or orders or volume. So we'll have a look at only 3 markets - firstly an exotic community market - KMD / VRSC. Verus is the home of the desktop wallet & Komodo is the original multi-chain blockchain project. Verus is a fork of Komodo, the address format is the same.

Scroll down to the bottom for VRSC
A button to go to the selected button is also half way down the screen - yes some UX / CSS adjustments to make...
The market data!

Step 5 - Update Coin Gecko Fiat Price

For the first integration with the centralized world, an example of the price feed being updated. This is a cyclical update on the middleware server. It fetches an update approximately every minute. From the above screenshot, to update to the latest fetched data, simply click on the banner!

KMD goes up a tiny bit like the good oscillating currency that it is

You will note the "INVERT" button. This will flip the market around. All orders are sellers, they are just selling at the opposite ends of the market! This is the underlying logic in the atomicDEX marketmaker software.

Step 6 - Invert Market

Not many VRSC sellers! This will increase, especially as I've deployed a coin specific interface for the community at https://vrsc.orderbook.live

What's Next

What's next for me is to do some live streaming using this interface. Creating orders with test coins and the community. Automation is a big feature of this orderbook app. It is designed for web communities to watch orders countdown in a reverse auction style contest.

Re-Using Passphrase

Instead of auto-generating a new passphrase on container start up, a plain text file can be mounted from the host to the container. The container has been built in such a way to auto detect the existence of a pre-shared passphrase!

Welcome to the first easy to explore full stack atomic swap project available for public use! All source code is open source with documentation to satisfy most curious souls. This page will be updated from time to time as the Komodo atomicDEX API comes out of beta and into full release stage. For now, everything is pretty stable in the core with not many changes to external facing APIs. This docker image serves visitors of the orderbook public app. On 30th April 2020, the orderbook demo is running version 0.7.8 backend available from the docker hub.

Getting Started With Atomic Swaps Using Docker

Firstly you will need docker installed to download and run the container in a reliable runtime environment across different operating systems. This is the overwhelming strength of docker from a software redistribution point of view.

Step 1 Pull AtomicDEX API & Middleware Image

docker pull coingateways/atomicdexorderbook:0.7.8

Step 2 Start AtomicDEX API & Middleware

The container supports sending assistive messages to a telegram bot. From telegram you will want to chat with "BotFather" and create a new bot. If you haven't got one, here is the create a bot guide from telegram. The bot will send you your multi wallet passphrase. This is super helpful for recording some wallet passphrases during development. An option to pass in a passphrase from a system file is available, which we will run through in another tutorial.

With a telegram bot

docker run -i -t -p 127.0.0.1:7780:7780 -e MMBOT_URL="http://localhost:7780"  -e BOT_TOKEN="XXXXXXXXXXXX" -e BOT_CHATID="XXXXXXX" -e BOT_USERNAME="XXXXX" -e ME_PUBLIC="true"  --name orderbook coingateways/atomicdexorderbook:0.7.8

Without a telegram bot

docker run -i -t -p 127.0.0.1:7780:7780 -e MMBOT_URL="http://localhost:7780" -e ME_PUBLIC="true"  --name orderbook coingateways/atomicdexorderbook:0.7.8

Step 3 Get Responses From Orderbook Middleware

The middleware is a web server that sits between the atomicDEX API RPC interface, and your browser. Browse to http://localhost:7780/coinsenabled and you will get a JSON response like the image showing 64 wallet addresses for different coins supported out-of-the-box.

Step 4 Explore Exposed API Endpoints

Try getting the BTC wallet balance with http://localhost:7780/getBalance?coin=BTC

Other URL endpoints require POST method, which is done through a web form or with a developer tool like insomnia or postman. Documentation for the other endpoints is waiting to be published after atomicDEX is released, here is a preview at the beta orderbook site.

Next Steps With AtomicDEX Orderbook App

The options are to attach a front end to it, but one comes inside the docker container! We will stop and redeploy the container in another tutorial to enable the front end web application.

Using the python electrum wallet sources, Komodo developer pbca26 was able to demonstrate to discord user zrtx goal on how to sign a raw transaction. The conversation included zCash protocol expert dukeleto, atomicDEX API & marketmaker developer artem.pikulin.

Electrum wallet is fully capable of signing correct kmd/kmd smartchain transactions and worked so far w/o any major issues. 99% of it's core code is inherited from zcash electrum wallet version. Here is the resulting gist (embedded below) from pbca26 to achieve the goal below:

komodo-cli createrawtransaction -> unsigned tx hash -> transaction lib of electrum-komodo -> signed tx hash

Most multicoin lite wallets can be used to enter a seed phrase (list of words) or a private key for a coin and be used for different coins. Sometimes, a new wallet will come out and offer features that you may want to try out. The wallet supports a private key for one type of coin for example, like bitcoin.

In the discord mining channel, it was asked:

is there a tool to get out of a komodo private key a BTC private key?

Here is the question.

Komodo team member Poly suggested to extract the key from a lite wallet, like what Verus Desktop & Chameleon Pay can do. Proof-of-Stake expert and LABS maintainer Alright from the Komodo team offered a python script for doing the task. It is included in the following gist below.

The requirements are to install base58 python package with pip install base58. Duke Leto from the HUSH private communications project and ZEC expert offered another private key conversion script with a useful tidbit of information:

ZEC/HUSH/etc have 2 prefix bytes, BTC/KMD only have 1

User CHMEX from the dexstats site has an online address converter to verify outputs of scripts, but it is for public addresses only - aka do not enter in private key or WIFs!

Komodo has released docker images of their work for wider developer appreciation and uptake. These are uploaded directly from the CI/CD developer pipeline.

Komodo Official Docker Images

From the discord chat, QA Manager TonyL made the announcement. A further example to create an assetchain, docker run -p 127.0.0.1:`ac_rpc_port`:`ac_rpc_port` -d --mount src=$HOME/.zcash-params,target=$HOME/.zcash-params,type=bind testblockchain

The HUSH blockchain team led by Duke Leto in 2019 removed the need for full node operators to download 900+ MB of redundant privacy proving keys (zcash params). The redundancy became available because HUSH3 is a newer "sapling" Komodo (& zCash) software fork. The previous "sprout" proving keys were not needed in the blockchain record for HUSH3, therefore removed from requirements to run the chain.

Recently in the Komodo discord, a journalist who has been "deplatformed" from video sites made an enquiry on how to make the p2p streaming available to his 150,000+ audience. Not long thereafter, a new channel was created to solve the situation of a government and social media platform conspiring to silence this well known journo.

announcement of p2p video sharing to counter government censorship of a local journalist
The new project for a p2p user app for video sharing

Community collaboration @ Komodo

Less than two weeks since the inception of this specialized project, the Komodo dev team and an enthusiastic community member had working prototypes of the "backend" of blockchain protocol & p2p sharing for the most part sorted out. This brought some user-adoption & UX tasks to rear it's ugly head.

Reading through the developer & testing dialogues, led me to confirm the current barriers needing solutions:

  1. Remove zcash param check
  2. Web streaming.
  3. P2P web streaming.

Web streaming (2) is easily solvable however it becomes centrally hosted and target for DoS. An acceptable problem to not-solve at this stage in development. zCash params was a huge limitation. The everyday user cannot be expected to download 1GB of redundant privacy keys to participate in file sharing, even if it is once.

HUSH collaboration with Komodo

I recalled Duke Leto's feat of making a 50MB download for his team's project instead of the 1GB redundant. Bandwidth efficiency & privacy in one. Asking him how much effort is required to remove zCash params altogether, and mentioning him in the relevant parts of the chat, I woke up this morning to this PR where the efficient developer has bypassed the zCash param loading circuitry.

Lowering The Barriers Continues

It's still early days in this p2p streaming application development. Less than 2 weeks since inception, and the Komodo public blockchains gain further efficiencies from ongoing community collaboration, lowering the technical requirements and setup for developer teams of the future. This new efficiency will likely make it's way into a docker container, lowering it's image size too.

Experience Crypto
© 2021 komodefi.com. Powered by Komodefi
chevron-down