First of all, I'm not pretending to speak for the LEDE team, however looking at the plain numbers presents a quite obvious situation.
Taking "Why no images generated for default D-Link DIR-600A1?" as an example (yes, some other 4 MB flash devices are a bit better than this particular specimen, the trend remains to be the same though).
D-Link DIR-600A1:
Backfire 10.03..............: 2293764 bytes Backfire 10.03.1............: 2949124 bytes Attitude Adjustment 12.09...: 2883588 bytes Barrier Breaker 14.07.......: 3276804 bytes Chaos Calmer 15.05..........: 3342340 bytes Chaos Calmer 15.05.1........: 3407876 bytes LEDE 17.01 release branch...: 3473412 bytes absolute firmware size......: 3735576 bytes maximum usable firmware size: 3538944 bytes
(all of these figures are for release images, including luci and a more or less identical feature set).
The erase block size for this (and most other) devices is 64 KB, so you now end up with 256 KB (== 4 erase blocks) free space, compared to 320 KB (== 5 erase blocks before). While this may look comfortable at a first glance, you have to consider that free space can only be assigned in (full) block size chunks, so once you touch the overlay partition at all, you already have one erase block in use (64 KB). Therefore the firmware creation tools used by LEDE enforce at least 3 erase blocks reserve for the overlay filesystem (that's where the maximum usable firmware size comes from, compared to the total size of the firmware partition). In other words, with 17.01 you'll only have 1 erase block (64 KB) before the hard limit, while 15.05.1 still gave you 2 spare erase blocks (128 KB) for your own use. On top of this there is the file system overhead needed for formatting to jffs2 as well (jffs2 does some light compression, but its fs header and log (more or less directory entries) need some space, then there is the hard requirement to keep some free space (in erase block == 64 KB chunks) for the garbage collection to work) at all times, reducing the usable free space even further. Around 25 KB are used by the configuration overlay immediately after firstboot, before you actually get a chance to configure anything.
Assuming pure statistics, what will the situation be in 12 or 24 months[1]?
No one has yet raised the suggestion to actually remove the hardware support for affected devices from LEDE's source repository. If you look deeper, you can still find full support for the Linksys WRT-54GL (4 MB flash, 16 MB RAM) in the repo, although support for devices with just 16 MB RAM had already been discontinued with Attitude Adjustment 12.09 and despite the fact that actually building a working up-to-date firmware for this device today is quite challenging[2].
This thread merely serves as a reminder for 'normal users', who expect to download and flash LEDE under the expectation to use it as a full featured replacement for the vendor firmware (including a webinterface, luci). Also keep in mind that luci is enabled for release builds, which means that it either fits into the firmware image (with the mandatory safety margin of at least 3 erase blocks) or no release images can be created for the affected devices[3], [4]. In my very personal opinion, this needs to be documented quite obviously, avoiding to raise false hope and expectations for (especially new) users.
Advanced users will obviously be able to delay the inevitable by quite some margin - depending on their use-cases and abilities to reduce LEDE's footprint below normal system requirements.
Despite the title, the RAM size is actually a much harder limit, affecting much more than just 4 MB flash devices. If you look through the commit log, you will notice quite some efforts to get opkg to use less RAM, but it's still a problem with just 32 MB RAM (even for normal operations, before touching opkg). Likewise you already are in trouble with sysupgrade and trying to flash a (larger) 6-8 MB firmware on a device with just 32 MB RAM, unless you really clean up services and loaded kernel modules manually beforehand, there is a high risk that you'll oom during the sysupgrade and brick your device for good (requiring bootloader/ tftp assistance to recover in the best case).
[1] switching from mach files to device tree (post 17.01) offers a potential to free up a little space in the kernel, but this isn't a whole lot and will probably be eaten more or less completely by upgrading to kernel 4.9+ (also keep in mind that the FDT file needs to be appended to the kernel image, which might or might not compress as well as the mach files before), so don't expect a significant positive effect from this switch.
[2] no webinterface, better no pppd, quite some custom configurations to get the kernel's runtime memory requirements as small as possible, disabling whatever is humanly possible (definately no IPv6, better no wireless either).
[3] with LEDE's pretty new feature of device specific rootfs images, it would technically be possible to decide between installing luci or omitting it on a per device basis, e.g. based on flash sizes, but support for something like this hasn't been implemented yet and would require quite some attention (both as in source patches and tagging of device classes) from interested parties. As implemented right now, the decision is binary - either the default (release) config (including luci) fits XOR building a firmware image for the affected device fails and no firmware will be available.
[4] I would expect that there already are a couple of 4 MB flash devices in the target list for which no release firmware can be built for 17.01, because of less ideal flash partitioning schemes chosen by the vendor (dropping free space below 3 erase blocks). Those are probably a minority, but given the close numbers, I'd be very suprised if there wouldn't be any affected.
