malipetek

← Knowledgebase

Solved

After one bad packet, every esptool call fails with an empty StopIteration

esptool esp8266 python generators retry serial debugging

This worked, and here is why.

The problem

A script drives esptool 4.x as a library over a lossy serial link, retrying each block on failure. After the first corrupted packet, every retry failed instantly, and the error had no message at all:

write 0x02c000: retry 1 (StopIteration)
write 0x02c000: retry 2 (StopIteration)
...
write 0x02c000: retry 6 (StopIteration)

The retry handler made it worse: it formatted errors with str(e).splitlines()[0], which raises IndexError on an empty message. That escaped the retry loop and the run died with list index out of range — at the same 12% mark, the first time looking like a random crash.

The recovery code drained the port, acked the stub and called esp.sync(), and the link really was fine. Nothing it did helped.

The fix

esptool reads replies through a generator, slip_reader(), stored as self._slip_reader and consumed with next(). When it hits a bad packet it raises FatalError — and a generator that has raised is finished. Every later next() raises StopIteration, forever, whatever the line is doing. Draining or flushing the port yourself does not help; only esp.flush_input() rebuilds it:

def flush_input(self):
    self._port.flushInput()
    self._slip_reader = slip_reader(self._port, self.trace)

esptool's own retry inside flash_block() has the same blind spot: it retries on FatalError, but the second attempt meets the dead generator and raises StopIteration, which it does not catch.

So recovery must rebuild the reader before re-syncing, and again after each failed sync:

drain(esp)              # read until the line is quiet
esp.flush_input()       # new slip_reader
for _ in range(5):
    try:
        esp.sync(); drain(esp, 0.2); esp.flush_input(); return
    except Exception:
        drain(esp, 0.2); esp.flush_input()

And never let error formatting raise inside a retry loop. Exceptions with empty messages are common here (StopIteration, struct.error, TimeoutError):

def why(e, n=60):
    lines = str(e).strip().splitlines()
    return (lines[0] if lines else type(e).__name__)[:n]

Test the recovery against a mock whose reader dies the way the real one does — a generator you close() — not against one that just raises once and keeps working. The simple mock passed while the real board failed.