Hiveon T17 latest: All pools dead: Restarts again and again: Damages hashboards

[evapowah]

This is Hiveon T17 latest, probably also other versions. When all pools are dead it enters a dead restart loop. It has a period of about 7 minutes in T17 machines. It does not check for “all pools dead” until the hashboards have been warmed up. It does not check for “all pools not dead” before shutting down hashboards. “I have soldered these boards by hand and it has made 10+ unneeded restarts in last 2 weeks.”

Is there a way to fix that firmware? Stock firmare either waits for “some pools not dead” before activating hashbords, or waits for “some pools not dead” before restarting. Also, stock firmware has a long timeout for “all pools dead” like 30 minutes. Can’t this be configures? Currently Hiveon T17 has no timeout, even short internet connection glitches of less than a minute can trigger a restart! And wear out hash boards.

Also, a selectable longer warm up time (frequency ramp up time), like 5 minutes would be a key feature for many machines. As it is found in latest T17e firmwares, which use constant temperature frequency ramp taking like 3 to 5 minutes, and also ramp down at shutdown.

Regards

[keaton_hiveon]

Thank you for your feedback, i passed it along to the devs

[evapowah]

More fedback:

• Upon automatic restart, hiveon T17 latest “miner status” sometimes freezes like that (machine still works):
  

  

• This is taken from “last error cause”. This is a system with a router behind a router. The main router became powered off at 21:40-somehting hours, leaving DHCP OK, LAN OK, but no DNS and no WAN.

2023-12-15 05:04:40 Shutdown miner
2023-12-20 18:11:46 FEE: can’t connect to 1.32:443
2023-12-20 18:11:51 FEE: can’t connect to 1.32:443
2023-12-20 18:11:56 FEE: can’t connect to 1.32:443
2023-12-20 18:12:01 FEE: can’t connect to 1.32:443
2023-12-20 18:12:06 FEE: can’t connect to 1.32:443
2023-12-20 18:12:11 FEE: can’t connect to 1.32:443
2023-12-20 18:12:16 FEE: can’t connect to 1.32:443
2023-12-20 18:12:21 FEE: can’t connect to 1.32:443
2023-12-20 18:12:26 FEE: can’t connect to 1.32:443
2023-12-20 18:12:31 FEE: can’t connect to 1.32:443
2023-12-20 18:12:36 FEE: can’t connect to 1.32:443
2023-12-20 18:12:41 FEE: can’t connect to 1.32:443
2023-12-20 18:12:46 FEE: can’t connect to 1.32:443
2023-12-20 18:12:51 FEE: can’t connect to 1.32:443
2023-12-20 18:12:56 FEE: can’t connect to 1.32:443
2023-12-20 18:13:01 FEE: can’t connect to 1.32:443
2023-12-20 18:13:06 FEE: can’t connect to 1.32:443
2023-12-20 18:13:11 FEE: can’t connect to 1.32:443
2023-12-20 18:13:16 FEE: can’t connect to 1.32:443
2023-12-20 18:13:21 FEE: can’t connect to 1.32:443
2023-12-20 18:13:26 FEE: can’t connect to 1.32:443
2023-12-20 18:13:31 FEE: can’t connect to 1.32:443
2023-12-20 18:13:36 FEE: can’t connect to 1.32:443
2023-12-20 18:13:46 FEE: can’t connect to 7.7:443
2023-12-20 18:13:54 FEE: can’t connect to 7.7:443
2023-12-20 18:14:37 FEE: can’t connect to 7.7:443
2023-12-20 18:15:12 FEE: connected to 18.84:443
2023-12-20 18:27:14 FEE: can’t connect to 1.32:443
2023-12-20 18:27:19 FEE: can’t connect to 1.32:443
2023-12-20 18:27:24 FEE: can’t connect to 1.32:443
2023-12-20 18:27:29 FEE: can’t connect to 1.32:443
2023-12-20 18:27:34 FEE: can’t connect to 1.32:443
2023-12-20 18:27:39 FEE: can’t connect to 1.32:443
2023-12-20 18:27:44 FEE: can’t connect to 1.32:443
2023-12-20 18:27:49 FEE: can’t connect to 1.32:443
2023-12-20 18:27:54 FEE: can’t connect to 1.32:443
2023-12-20 18:27:59 FEE: can’t connect to 1.32:443
2023-12-20 18:28:04 FEE: can’t connect to 1.32:443
2023-12-20 18:28:09 FEE: can’t connect to 1.32:443
2023-12-20 18:28:14 FEE: can’t connect to 1.32:443
2023-12-20 18:28:19 FEE: can’t connect to 1.32:443
2023-12-20 18:28:25 FEE: can’t connect to 1.32:443
2023-12-20 18:28:30 FEE: can’t connect to 1.32:443
2023-12-20 18:28:35 FEE: can’t connect to 1.32:443
2023-12-20 18:28:40 FEE: can’t connect to 1.32:443
2023-12-20 18:28:45 FEE: can’t connect to 1.32:443
2023-12-20 18:28:50 FEE: can’t connect to 1.32:443
2023-12-20 18:28:55 FEE: can’t connect to 1.32:443
2023-12-20 18:29:00 FEE: can’t connect to 1.32:443
2023-12-20 18:29:05 FEE: can’t connect to 1.32:443
2023-12-20 18:29:14 FEE: can’t connect to 7.7:443
2023-12-20 18:29:24 FEE: can’t connect to 7.7:443
2023-12-20 18:30:13 DNS FAILED
2023-12-20 18:30:37 FEE: can’t connect to 7.7:443
2023-12-20 18:31:09 FEE: connected to 34.18:443
2023-12-21 21:43:52 FEE: can’t connect to 1.34:443
2023-12-21 21:43:57 FEE: can’t connect to 1.34:443
2023-12-21 21:44:02 FEE: can’t connect to 1.34:443
2023-12-21 21:44:07 FEE: can’t connect to 1.34:443
2023-12-21 21:44:12 FEE: can’t connect to 1.34:443
2023-12-21 21:44:17 FEE: can’t connect to 1.34:443
2023-12-21 21:44:22 FEE: can’t connect to 1.34:443
2023-12-21 21:44:27 FEE: can’t connect to 1.34:443
2023-12-21 21:44:32 FEE: can’t connect to 1.34:443
2023-12-21 21:44:37 FEE: can’t connect to 1.34:443
2023-12-21 21:44:42 FEE: can’t connect to 1.34:443
2023-12-21 21:44:47 FEE: can’t connect to 1.34:443
2023-12-21 21:44:52 FEE: can’t connect to 1.34:443
2023-12-21 21:44:57 FEE: can’t connect to 1.34:443
2023-12-21 21:45:02 FEE: can’t connect to 1.34:443
2023-12-21 21:45:07 FEE: can’t connect to 1.34:443
2023-12-21 21:45:12 FEE: can’t connect to 1.34:443
2023-12-21 21:45:17 FEE: can’t connect to 1.34:443
2023-12-21 21:45:22 FEE: can’t connect to 1.34:443
2023-12-21 21:45:27 FEE: can’t connect to 1.34:443
2023-12-21 21:45:32 FEE: can’t connect to 1.34:443
2023-12-21 21:45:37 FEE: can’t connect to 1.34:443
2023-12-21 21:45:42 FEE: can’t connect to 1.34:443
2023-12-21 21:45:47 FEE: can’t connect to 1.34:443
2023-12-21 21:45:52 FEE: can’t connect to 1.34:443
2023-12-21 21:45:57 FEE: can’t connect to 1.34:443
2023-12-21 21:46:02 FEE: can’t connect to 1.34:443
2023-12-21 21:46:12 FEE: can’t connect to 7.7:443
2023-12-21 21:46:22 FEE: can’t connect to 7.7:443
2023-12-21 21:46:31 FEE: can’t connect to 7.7:443
2023-12-21 21:49:14 DNS FAILED
2023-12-21 21:50:06 DNS FAILED
2023-12-21 21:50:43 All pools are dead, restarting
2023-12-21 21:50:58 DNS FAILED
2023-12-21 21:51:49 DNS FAILED
2023-12-21 21:52:40 DNS FAILED
2023-12-21 21:53:31 DNS FAILED
2023-12-21 21:54:23 DNS FAILED
2023-12-21 21:54:27 Shutdown miner
2023-12-21 21:58:42 All pools are dead, restarting
2023-12-21 21:59:27 DNS FAILED
2023-12-21 22:00:19 DNS FAILED
2023-12-21 22:01:11 DNS FAILED
2023-12-21 22:02:03 DNS FAILED
2023-12-21 22:02:27 Shutdown miner
2023-12-21 22:06:28 All pools are dead, restarting
2023-12-21 22:07:10 DNS FAILED
2023-12-21 22:08:06 DNS FAILED
2023-12-21 22:08:58 DNS FAILED
2023-12-21 22:09:50 DNS FAILED
2023-12-21 22:10:13 Shutdown miner
2023-12-21 22:14:14 All pools are dead, restarting
2023-12-21 22:14:56 DNS FAILED
2023-12-21 22:15:49 DNS FAILED
2023-12-21 22:16:41 DNS FAILED
2023-12-21 22:17:33 DNS FAILED
2023-12-21 22:18:00 Shutdown miner
2023-12-21 22:22:01 All pools are dead, restarting
2023-12-21 22:22:43 DNS FAILED
2023-12-21 22:23:35 DNS FAILED
2023-12-21 22:24:27 DNS FAILED
2023-12-21 22:25:19 DNS FAILED
2023-12-21 22:25:47 Shutdown miner
2023-12-21 22:29:48 All pools are dead, restarting
2023-12-21 22:30:30 DNS FAILED
2023-12-21 22:31:22 DNS FAILED
2023-12-21 22:32:14 DNS FAILED
2023-12-21 22:33:06 DNS FAILED
2023-12-21 22:33:34 Shutdown miner
2023-12-21 22:37:57 All pools are dead, restarting
2023-12-21 22:38:39 DNS FAILED
2023-12-21 22:39:31 DNS FAILED
2023-12-21 22:40:22 DNS FAILED
2023-12-21 22:41:13 DNS FAILED
2023-12-21 22:41:41 Shutdown miner
2023-12-21 22:46:01 All pools are dead, restarting
2023-12-21 22:46:43 DNS FAILED
2023-12-21 22:47:36 DNS FAILED
2023-12-21 22:48:28 DNS FAILED
2023-12-21 22:49:24 DNS FAILED
2023-12-21 22:49:47 Shutdown miner
2023-12-21 22:53:48 All pools are dead, restarting
2023-12-21 22:54:30 DNS FAILED
2023-12-21 22:55:21 DNS FAILED
2023-12-21 22:56:12 DNS FAILED
2023-12-21 22:57:03 DNS FAILED
2023-12-21 22:57:34 Shutdown miner
2023-12-23 12:15:39 All pools are dead, restarting
2023-12-23 12:19:20 Shutdown miner

[evapowah]

One more feedback:
When one hashboard is disconnected or not booting OK the custom frequency settings for that board are lost. Often requiring one or more additional power cycles for setting up .

[evapowah]

One more feedback:
There is yet another bug that allows an erratic pool application (in this case I discovered it with ultimateproxy beta) with erratic communication to make the firmware think that there are no more pools available, when in fact there are. This causes an unneeded re-init.

[evapowah]

More feedback.
When the DNS server goes down, as when restarting the routers, sometimes Hiveon copes with it. sometimes it does not. When it does not, it keeps restarting and mining for a few minutes without showing the
2024-05-11 00:01:14 driver/driver-btm-api.c:3018:dhash_chip_send_job: Version num 4.
text.

It’s an scenario with 2+ possible internet routes, with some of the alternative DNS severs that only work for some of the routes. It could be too NTP servers not working for some of the internet routes. There is a main gateway which acts as DHCP and DNS and knows which DNS to redirect to, but when the main gateway gives timeouts after the gateway DNS server fails for some time, rater than retrying on the gateway as DNS server Hiveon seems to try the DNS IPs on its own.

It did it 20 times today with freshly fixed hashbosrds. Loss of time? I’m considering stock firmware, leaving Hiveon only for testing. “I hope the -functionality- is not into newer versions”. But we are still paying devfee on old versions. You could consider removing devfee for old versions if they are no longer going to get revised.

EDIT: Oh, I also tried to access system through ssh to analyze the code and try to fix the scripts at least to prevent a dead restart loop. But I was unable. There is no option to enable ssh in hiveon and the button to access ssh through hiveos farm seems to be missing for this firmware. VNISH has all this, except that it starts too fast for T17 and damages hashboards that way.