Limitations on connections in ISP equipment (was LEDE packet per second performance versus other firmware)

Hmm.. without your special TOR node or whatever, is there still a connection table issue? Are there hundreds of port 53 DNS lookups? You could at least eliminate those using your own DNS cache. IPFire uses "unbound" for DNS caching:

https://wiki.ipfire.org/dns/start

It should be handing out its own address as DNS server when doing DHCP so hopefully it's not the case that all your LAN machines are individually hitting external DNS

Beyond that, your next bet is to have everyone use your router as a squid cache for web access.

This lets you both control web access to some extent, and reduce connection count as squid will reuse connections. "server_persistent_connections" is on by default.

What you have to do though is to configure all the LAN clients to use the proxy. Various ways to do that, depends on which browser and which OS is in use.

Probably details of how to configure IPFire should be questions for the IPFire message boards, they will have that expertise.

If you wouldn't mind, can you do a dslreports speed test on your system during "quiet" time and post the results here? I'm interested to see how well IPFire's QOS works and how well this old hardware handles its routing task.

Thanks

Sure. Least can do is give a speed test later if would like.

IPFire Squid proxy is easy. Tested when first installed. There is a transparent proxy option that pushes everything through (and can exclude IP's or MAC's or whatnot, ect).

DNS was never connection hog previously with Dnsmasq or now with Unbound. Both just send queries locally to a recursive Bind server which goes external. It might have even made more sense to send everything to the local DNS server instead of going through Unbound and router/ IPFire at all. Didn't though because Unbound caches. But Bind probably caches itself too. Still though not big decision at all and can change that easily.

No NAT table saturation issue at all without the Tor bridge. That being said at about 1,200-1,500 connections in NAT table normally and can run the Tor bridge okay with 300 TCP and 300 UDP connection limit (not sure if UDP is used at all by bridge). So there isn't much room either way but is okay for now. Bridge may just have to be tabled or more likely stay limited as it is a complete side project and causes me trouble with PCI compliance anyways every 90 days to get exceptions. The neat thing about IPFire though is that was easily able to only allow incoming connections to bridge from Iran, Turkey, China and anyone else that choose to include thus limiting connections solely to those trying to avoid specific governments.

Bridge on: https://www.dslreports.com/speedtest/29113738

when I click that link I get a mostly blank page, no graphs, strange. One thing I do notice is that you get a C for bufferbloat. This makes me think you could probably use some improved QoS settings. Unfortunately I don't know much about how IPFire configures QoS but you might ask on the IPFire forums how to set up their QoS to ensure reasonable buffer management.

@schnappi try installing LEDE + SQM packages and repeat the same test, you'll see a great improvement.

Or if the current IPFire distro is working fine for other purposes, just try tuning the QoS settings. I can now see the graphs and things, and it seems like you probably need to set your max bandwidth up and download speeds better. The results you have for bufferbloat indicate that your router is probably not the bottleneck and somewhere else is doing the buffering. Set speeds something slightly less than what you really have, especially for download, set your speed to around 85% of tested speed, and then bump it up a few percent at a time, for upload set your speed to around 95% of tested. IPFire uses fq_codel and should get perfectly fine bufferbloat scores if you have the speeds set appropriately.

Or try LEDE with cake + pice_of_cake as SQM, as far as I know works better than fq_codel :wink:

Main LEDE docs recommends cake instead of fq_codel https://lede-project.org/docs/user-guide/sqm
(that it does't mean that fq_codel is not good for SQM).

A few things to note. First the IPFire community is not nearly as helpful as LEDE. Had two issues and eventually figured out all on own. Forums were not very helpful and one issue didn't receive reply after days. Second IPFire GUI is not intuitive. Some things appear broken in the GUI but they actually are not, they just have to be enabled/ setup non-intuitively. Third IPFire states that they do not believe in "security through obscurity" but SSH runs on 222 by default which is annoying if not expecting this but really not a big deal at all. Forth turning on QOS with defaults resulted in receiving an "A" for bufferbloat and "quality" on the DSLReports speedtest. Assume these are both good things. Fifth reviewing currently online devices is not nearly as good as Tomato or probably LEDE, also cannot easily check bandwidth per device if this is one's thing. Finally despite the above all things considered would absolutely recommend IPFire to anyone.

Would also add that would have liked to try LEDE first on x86 hardware but had a somewhat difficult time finding documentation to do so. IPFire was easier in this regard (maybe only because it was built for hardware instead of routers). Granted am not the most technical person and have no formal training but can usually figure things out. This is the sole reason why did not try LEDE first on x86 hardware.

Currently using static ATT static IP but have not yet tested if NAT table limit on ATT device applies when using a static IP. When contacted about the NAT table limit issue ATT strongly denied that the ATT device NAT table gets filled when using IP Passthrough (without a static IP).

Static IP did not bypass ATT device NAT table limit. Signed contract for dedicated fiber connection with service level agreement. Problem solved since installation. Still disappointing as other/ most providers do not have such small NAT table limits even when custom equipment is required for residential or small business connections.

Thanks for help and assistance. Speaks highly of LEDE community.

What is appx cost of fiber w sla? Also do you now just have direct connection to Ethernet jack w/o any cpe router equip?

What is name of this svc?

Was going to wait until first bill but would be waiting a month. $997 plus taxes and fees minus $300 for 12 months.

Do not know how it is setup. Was not present during install. Building maintained closet on every floor is where conduits are and where internet comes in. Instead of putting router there like neighbour who shares one of my other floors put router in office and connect service provider line to building closet patch panel and have a large Cisco switch to deal with rest of patch panel which in turn connects to large switch inside office. Left a note telling install team to hook up to a specific port on patch panel.

Assuming it is an ethernet cable. Router is getting IP via DHCP. In note to install team asked about upstream equipment. They left a note saying there was none that needed to worry about.

Price is excessive but learning and running some services is a nice hobby that gives mind a break by being challenging but not all that stressful while supporting good causes around the globe.

Yeah, that's a lot, about 10x the cost of Uverse fiber with the limited router, so I'm guessing those prices will wind up converging eventually, but for now it seems like a market segmentation technique: set people up with limited NAT tables and things, and then upsell the most demanding customers on a thing that costs 10x as much. The world is a weird place in telecom right now. Used to be that a gigabit fiber would be something like $10,000 a month and of course telecom companies are loath to compete with themselves on that ... but ultimately everyone wants a lot of speed, even not-tech-savvy people are signing up for hundreds of megabits or gigabit where available, so we'll see as time progresses.

EDIT: it was only a few years ago that I called up a local provider and they tried to sell me a T1, at a time when I was already getting 60/3 docsis. I just laughed. Eventually they did offer me 10/10 metro ethernet, but it was $200/mo which was about 3 times the 60/3 docsis. Today I have gigabit fiber for about $100/mo but limited by this limit that you've mentioned here, but so far for my purposes it's ok.

Nothing has much changed. Quote for a gigabit dedicated connection likely would have been the almost the exact base price you referenced (purchased 100 Mbps fiber dedicated symmetrical). This absolutely is a sly and deceptive tactic but what business does not practice the same tactics in their own respective fields? Although going beyond this telecom providers do get public subsidies in multiple forms from states to expand fiber networks that they then proceeded to sell in its full capacity solely to those who can afford it. Again though who wouldn't act similarly in their own respective field? The issue is state and federal agencies foolish enough to give subsidies for expanding fiber connection without oversight. Many, including myself, would argue the solution is no subsidies at all for expanding high speed networks. Let the market decide where fiber networks end up instead of subsidizing permanent networks that are sold (more logically correct term would be to use rented) at high prices.

1 Like

thats mostly the case. and the effect is a huge gap between wealthy/dense and rural areas.

markets dont solve technical/social problems. only corporate ones (money).

In most places in the US there really isn't a functional market in internet providers. There is usually 1 locally granted monopoly, often a cable company.

In any case, discussion of the politics of large businesses who have government subsidies etc gets pretty far off topic. Suffice it to say that there are lots of reasons why most people can't plug into a high speed fiber today, and many of them are social, regulatory, tax, legal, and etc and have nothing to do with technological limitations. Hopefully in the future we'll see more people getting better and cheaper internet service, but don't hold your breath, it'll be decades.