Ethminer: Improvements proposal

[AndreaLanfranchi]

I would like to start a poll about possible introduction of new features (or rivise existing ones) in an effort to improve efficiency and usability.

Let me start

• Return on primary pool: the recent introduction of "-P" argument (over deprecated -S and -SF) have brought a high level of freedom in setting the pools ethminer can mine on. Specifically you can have 1, 2 or more and, more important, every pool definition is autonomous: this means I can have one pool mining ether and the second set to mine UBIQ (just to make an example). Despite this behavior is desirable we have _lost_ the concept of "Primary" vs. "Failover". At actual release level, ethminer before leaving a pool tries to reconnect to all it's available ip addresses but, when the switch to another pool occurs, that new connection stays active until a fatal error eventually occurs (timeouts, lost connection are basically the only two). In such case we will never return to the first pool unless the failing one is the last of the list. A quick and dirty way to address this is to consider the first pool of the list as "Primary" and, when switching to an alternate pool, begin to count a configurable amount of time which, when elapsed, will bring the connection back to primary (if available for connection).

Edit Jun 12, 2018 : see PR #1239

Your move now to add more ...

[invidtiv]

Thank you for this pool

I would like to see an option to turn off / on a GPU while mining, be means of a hotkey [0..9] [A..H] , in some troubleshooting of mining rigs, this would be helpful.

[chfast]

• Device abstraction: Both CUDA and OpenCL miners should return lists of the description of available devices, ordered by PCI id. Then the lists should be merged. The configuration can be applied to each device independently.

Edit 2018/11/27 Achieved in PR #1704

[SnowLeopard71]

These suggestions are for making _ethminer_ a better daemon:

• write PID to file using process name. (ie if etherminer renamed to etherminer-amd, write PID to /var/run/etherminer-amd.pid)
• option to output to file instead of stdout. default to using /var/log/process-name.log if not specified
• use config file for all options and have command-line act as overrides

[AndreaLanfranchi]

@SnowLeopard71

points 1 and 2 can be already done wrapping ethminer in a proper start bash

#!/usr/bin/env bash
# -----------------------------------------------------------------------------
# File      : miner.sh
# Project   : https://github.com/AndreaLanfranchi/ethereum_nvidia_miner
# Author    : Andrea Lanfranchi ([email protected])
# -----------------------------------------------------------------------------
# Description
# Starts / Stops /checks mining process
# -----------------------------------------------------------------------------

export DISPLAY=:0

# Here you specify the start cmd of your favorite miner.
# Keep as many as you want but remember to have only one uncommented at time
#
# start_lbl     keeps the executable name of your miner
# start_cmd     keeps the whole invocation command of your miner
#               you may want to refer to settings.conf for
#               a list of variables you may use
# stop_sig      keeps the signal to be used for killing miner process
#

# <!-- Begin Editing Area -->

# Ethminer
start_lbl="ethminer"
start_cmd="/home/prospector/ethereum-mining/ethminer/build/ethminer/ethminer -U -P stratum+tcp://[email protected]:4444 --report-hashrate --cuda-streams 2 --api-port 3333 --exit"
stop_sig="2"

# <!-- End Editing Area -->


# Do not change anything below this point unless you know what you're doing

main_status()
{

    if [ -s ~/miner.pid ];
    then
        pid=`cat ~/miner.pid`
        kill -s 0 $pid >/dev/null 2>&1
        if [ "$?" = "0" ];
        then
            printf "\\n%s is running with process id %s\\n\\n" $start_lbl $pid
            ps -p $pid -o pid,comm,etime,args
            echo
            exit 0
        else
            printf "\\n%s with process id %s is not running\\n\\n" $start_lbl $pid
            exit 1
        fi;
    else
        printf "\\n%s appears to be stopped\\n\\n" $start_lbl
    fi;

}

main_start()
{

    if [ -s ~/miner.pid ];
    then
        pid=`cat ~/miner.pid`
        kill -s 0 $pid >/dev/null 2>&1
        if [ "$?" = "0" ];
        then
            printf "\\n%s is already running with process id %s\\n\\n" $start_lbl $pid
            exit 0
        fi;
    fi;

    printf "\\nStarting %s ...\\n\\n" $start_lbl
    touch ~/miner-gpu.log

    # Enable persistence mode and apply power limit
    sudo nvidia-smi -pm ENABLED >> ~/miner-gpu.log 2>&1
    bash ~/nvidia-overclock.sh -pl >> ~/miner-gpu.log 2>&1

    # Set environment variables for EthMiner (mainly needed only for AMD cards)
    export GPU_FORCE_64BIT_PTR=0
    export GPU_MAX_HEAP_SIZE=100
    export GPU_USE_SYNC_OBJECTS=1
    export GPU_MAX_ALLOC_PERCENT=100
    export GPU_SINGLE_ALLOC_PERCENT=100

    # Set CUDA specific variables
    export CUDA_DEVICE_ORDER=PCI_BUS_ID

    # Disable colors
    export NO_COLOR=1

    # Start the miner
    nohup $start_cmd >> ~/miner-gpu.log 2>&1 </dev/null & echo $! > ~/miner.pid & sleep 1

}

main_stop()
{

    if [ -s ~/miner.pid ];
    then
        pid=`cat ~/miner.pid`
        kill -s 0 $pid >/dev/null 2>&1
        if [ "$?" = "0" ];
        then
            printf "\\nKilling %s with process id %s \\n" $start_lbl $pid
            read -p "Are you sure [Y/n] ? " -n 1 ; echo
            if [[ $REPLY =~ ^[Yy]$ ]];
            then
                kill -s $stop_sig $pid >/dev/null 2>&1
                exit $?
            fi;
        else
            printf "\\n%s is bound to process id %s but the process is not running\\n" $start_lbl $pid
            read -p "Do you want to try with 'killall $start_lbl' [Y/n] ? " -n 1 ; echo
            if [[ $REPLY =~ ^[Yy]$ ]];
            then
                killall -s $stop_sig $start_lbl
            fi;
        fi;
    else
        printf "\\nCould not find %s process id %s \\n" $start_lbl $pid
        read -p "Do you want to try with 'killall $start_lbl' [Y/n] ? " -n 1 ; echo
        if [[ $REPLY =~ ^[Yy]$ ]];
        then
            killall -s $stop_sig $start_lbl
        fi;
    fi;

}


case "$1" in
-status)    main_status
            ;;
-start)     main_start
            ;;
-stop)      main_stop
            ;;
-help)      clear
            echo " Name "
            echo "      miner.sh"
            echo
            echo " Synopsis"
            echo "      miner.sh -start | -stop | -status | -help"
            echo
            echo " Description"
            echo "      Starts / stops / checks mining process"
            echo
            echo " Options"
            echo "      -start  to start the miner"
            echo "      -stop   to stop the miner"
            echo "      -status to check the status"
            echo "      -help   displays this text"
            echo
            ;;
*)          echo
            echo "Usage : miner.sh -start | -stop | -status | -help"
            echo
            exit 2
            ;;
esac
exit 0

[xbkingx]

Some quality of life features I'd like to see:

• Color output.
• Separate config files for pools, compute devices (CPU, GPU, and (ugh) ASICs), and general settings. Ideally these could exist as separate files or a single merged file. Again, I think xmr-stak has a good setup.
• Responsive tweaking (e.g.-press 'p' while mining to increase cuda-block-size in x increments, x defined in config).
• Calculation of ETH/hr (or min, sec, etc) based on current price. Calculation of ETH/Watt.
• Allow adding pool payout levels to the config, so you can track % remaining and time until payout (based on either polling pool balance or manual entry at a couple time points).
• Switch 'profiles' without closing the miner - basically drop intensity if working on the machine or raise it with a single key press.
• Autoupdate - check for new version at launch, get confirmation, download, backup old version, extract new version.

Most of these could be done with batch files of varying complexity, but I think these features would make deploying to multiple machines easier and mining on a machine that's actually used more tolerable. I realize it's probably useless now, with the ASICs coming and PoS on the horizon (haven't kept up on the developments there), but those small 1-5 machine miners kinda help with the whole decentralization part and it would be nice to encourage more people to bolster the network.

[AndreaLanfranchi]

@xbkingx
aside the fact colored output is already in place ... all others are requests more addressed to a "revenue-optimization-system" rather than a miner. Are completely out of scope.

[xbkingx]

@AndreaLanfranchi
Everything is outside the scope besides "be faster/more efficient", "don't lose connection", and "don't crash". Your suggestion in the OP should be considered a bug. Running permanently on a failover system is like running off a backup flash drive after an HDD failure.

Separate config files and live inputs are found in other miners, so clearly not everyone shares your definition of miner scope. Colors are to console output, as emoji are to messaging (haha), so I'm surprised they made the cut.

Again, some small hobby miners probably prefer a single hybrid program (I know I would have), if for no reason other than the fewer pieces of crypto software you install the safer. You're entrusting the reputation of ethminer with some random person's front end around novices. They'll chalk up the 10% lower hash rates to ethminer requiring too many tweaks (not the added dev fee) and get scammed or look elsewhere.

When someone asks, "How do I mine ETH?" you want to say, "Pick a pool and just download ethminer." That will scare away a lot fewer people than talking about the details of stale shares, CUDA cores, forking the repo, and the fine art of managing 25 different bat files while tweaking.

Not a big deal, hence "Quality of life" heading. New user adoption is about to drop off a cliff anyways, so you guys can decide if bringing in new users is worth the work.

[AndreaLanfranchi]

@xbkingx I'm taking some time to try respond (just for fun) to your requests.
Please take into account this is my personal opinion and does reflect only my point of view. I am not the ambassador of this project nor I pretend to speak in name of other contributors.

• Color output.

As already mentioned colors in console output are there (I guess since long time now). If it was for me I would have not spent a single minute implementing them. As colors are perceived only by human eye I consider idio... errmmm ... not so clever to stare at you screen watching line of messages roll away eventually shouting "Hurray !" each time a green "Accepted" line passes. Even for hobbysts.

• Separate config files for pools, compute devices (CPU, GPU, and (ugh) ASICs), and general settings. Ideally these could exist as separate files or a single merged file. Again, I think xmr-stak has a good setup.

Not so clear what you're asking for here. Better say you're making some confusion. Ethminer is GPU only (maybe ... remotely maybe will bring back CPU computing) but ASIC's have nothing to do with a software like this is : they're a completely different planet as they're primarily hardware !
More : all ethminer needs to function is a connection to a pool (or many of them) to receive jobs, do the computational work and, eventually, push back the solution(s) found. Although we might agree it could be useful to persist all the CLI arguments in a file (which is btw nothing more than to store all CLI arguments in start batch) there are no additional configuration values useful for different pools. Ethminer cannot perform overclocking/undervolting or control fans: afaik nor the well known Claymore's miner can do that (at least not on Nvidia and only with limited features on AMD).

• Responsive tweaking (e.g.-press 'p' while mining to increase cuda-block-size in x increments, x defined in config).

From my personal pov this is a nonsense or even detrimental. Aside the fact that such value changes require the miner to restart each time (as the device must be reset, new memory must be allocated and so on) I consider the initial job to find the best possible running values a one time job a user has to perform with some degree of knowledge. Once it's done ... it's done. Hopefully what we all expect from a miner is to run flawlessly for hours, days, weeks or months without the burden to readjust values each time. This, believe me, is the most effective way to make profits in mining environment: continuity !

• Calculation of ETH/hr (or min, sec, etc) based on current price. Calculation of ETH/Watt.

Ethminer is not (only) an Ether miner ... is an ETHASH (meant as algorythm) miner. You can mine ETH, UBQ, PIRL, MUSIC ... whichever blockchain'ed coin which rely on the PoW algo ETHASH.
Thus calculation of what ? Price of what ? Eventually price from where ?
Regarding the effective production per Watt the only thing we could compute is how many hashes per watt absorbed by each gpu ... but aside the fact some GPUs do not expose the effective consumption (say NVIDIA Gtx 1050 Ti) a GPU cannot function without a motherboard, a cpu, some ram ... thus the consumption must be considered of the whole machine, but, unfortunately we cannot know that.

• Allow adding pool payout levels to the config, so you can track % remaining and time until payout (based on either polling pool balance or manual entry at a couple time points).

Each pool has different policy: some pay stale shares, some do not, some have a reward system based on PPLNS, some on RBPPS, there is even ethpool.org which has a pretty unique predictable system where you get all the reward of a mined block if you're the one on top of a list of most active contributors.
It would impossible for ethminer to handle with all such systems. But there is more : value rewarded per share vary upon other parameters like, for example, difficulty: some pools have constant diff, some have changing. Not to count pool fees. What if then you change your payout threshold on the pool and "forget" to set this value in your config files ? Or do you mean ethminer should be able to pool all different flavours of API interfaces (if any) provided by each pool ?
Time to next payout : if you'd know how exactly mining works you'd also know there is a sensible "luck" factor. In extreme summary: mining is like extract random numbers from an immense bingo bag and only few are valid. On the long time average you can estimate (based on your current hashrate) you will pick the right ones at a certain rate but when examining a short period of time (which is generally speaking what most hobbyst choose as their payout) luck factor is very very relevant. You may pick no valid numbers for a sensible amount of time.

I reiterate ... wether your' an hobbyst or a pro ... an adeguate, even minimal, business plan must be studied before starting your activity and this include, among others, the correct (for you) choice of the coin, the pool, the miner and everything might be useful to maximize your profit. Then start mining and let machines do their job.

• Switch 'profiles' without closing the miner - basically drop intensity if working on the machine or raise it with a single key press.

Very same argument as interactive tweaking mentioned above.

• Autoupdate - check for new version at launch, get confirmation, download, backup old version, extract new version.

This is not a good idea indeed. It's not always true a new release (of a miner) is more efficient FOR YOU than a previous one: exactly like installing, for example, new GPUs drivers' release might crash your system. It's a matter of environment. Say you have an "old" Gpu which handles well with an old version of driver ... would you change driver and pay to cost of seiing your hashrate drop ? I believe not !
As we're talking about a world where we push machines to their limits and users are very fast about complaining the fact in release xyz they go x hashrate while with release abc they got x-1 hashrate : it would be a complete disaster to autoupdate. Tons of furious users clogging these pages with new issues opened.

In conclusion ... I, as a humble contributor, will never spend my time (for free) on a project aimed to alleviate miners (wether occasional or pro) from the duty of their homework. Nor I will endorse any form of auto-update.
This is not a commercial software project where we guarantee any sort of SLA (read the license) and, personally, I do not want to fall in the same error I've witnessed too many times while developing in other commercial areas : "customers" always want an all-in-one solution as they're not able to properly use all the tools a pc has to offer. But in that case they're customers and thus paying: as long as they pay for their foolishness or laziness ... well ... the more the better for my pocket.

Here, instead, we do not get any money for our job nor we endorse any fee. My only gratification is from users (if any) who once may say "Good job. Thank you !"

Best.

_This is my personal opinion and does necessarily reflects the sentiment of the project's coordinators or other contributors._

[AndreaLanfranchi]

• Stratum mode autodetection : at present it's up to user to enforce the right stratum mode (standard, eth-proxy, ethereumstratum) with stratum/1/2. Would be helpful if we could autodetect stratum mode.

Edit 2018/07/11 Feature added #1333

[AndreaLanfranchi]

• Low intesity mode : appears to be a feature highly requested for users who mine and work on the same pc.

[AndreaLanfranchi]

• Configurable Response Timeout : actually response timeout to shares submission is hardcoded to 2 seconds. This value is reasonable and have strong ratio but may have impact when internet connection is not dedicated to the mining activity. Wether other machines on the same connection may request huge downloads or non-professional links may be in use it may be useful to adjust this parameter accordingly.
  This value should be the same for any response we're waiting from pool server thus including connection timeout.

Already committed in #1086

[Starkot]

My suggestions:

• Whatsnew: what is the difference between dev5 and dev6? And from dev7?
• How much time has passed since the last green "Accepted"
  (e.g. press 'a' while mining to show).
• The ability to terminate the miner with a specific exit code in the following cases:
  No "Accepted" for more than XX minutes.
  Just after XX minutes/hours.
  The hashrate drop below XXX for XX minutes.
  The GPU temperature rose above XX degrees.
  etc.

[chfast]

Is this still relevant?

[Starkot]

I can do without.

[AndreaLanfranchi]

Many requests already achieved.
Closing.

Feel free to open new requests as single topics.