LEDE Table of Packages: Good or bad?

Talk about Documentation
81

[quote="tmomas, post:76, topic:123, full:true"]Which information would be dropped?[/quote]Different version, and any other possible difference between packages in a stable branch and packages in snapshots (other dependencies, size differences, other architectures)

[quote]Why is it important to have the dropped information?[/quote]Because if you show only snapshot, all people on stable releases might not see the right info for their packages.

[quote]Who would use the table of packages to answer which question?
Or in other words: What would be the usecase of the ToH?[/quote]
The main target imho is people planning for a new system (LEDE or firmware-not-yet-decided), so they can see what LEDE firnware can offer.
Then they can use the information with either the Image Builder or installing packages in a device.

Secondly people looking about more info for a specific package (tutorials or whatever), although this will be done by different table views than those used by the first usecase.

In both cases having the most accurate data about the LEDE release's packages is the bare minimum they expect.

[quote]Exactly. And all that missing information that is displayed in those dummy pages doesn't give a good picture. We should avoid having 4000 half done package-pages, and rather prefer 100 well done ones that we then link to.[/quote]Nice logical fallacy right there. We aren't choosing between 100 well done pages and 4000 dummy ones.

We are choosing between 4000 dummy ones (plus a dozen I import from OWRT wiki) and nothing (plus a dozen I import from OWRT wiki but will have to go in some other category of user guide, in a similar way than current OWRT wiki).

I think that having the pages available at all will inspire more potential contributors than not having them. OWRT wiki does not have them (nor the ToP) and it has little info about packages at all (apart from some VPN and 3G and multiwan stuff).

Please send a pull request to the maintainer of the gh plugin.
https://www.dokuwiki.org/plugin:gh

I signed up for LEDE, not for fixing shitty Docuwiki plugins.

:expressionless:
I'm starting to regret the decision to use Docuwiki.

Drop that link then, it will be rendered better with a direct copy-paste.

82

Take a look: https://wiki.lede-project.org/pkgdata/adblock-snapshot

:slight_smile:

83

Not seeing any difference to how it was before.

84

Reload the page and/or delete your browsers cache.
There IS a change :slight_smile:

85

Hm, found a better solution to keep all in a single package list.
I'm going to add release-specific fields to the same package txt, like I did with the version-short field.

Then the ToP of each release will show different fields from the same package txt.

86

Just opened it in the incognito mode, thank you,[quote="tmomas, post:84, topic:123, full:true"]
Reload the page and/or delete your browsers cache.
There IS a change :slight_smile:
[/quote]

extra line breaks are gone!

87

loaded another batch of package txt in the wiki's media manager.

Also the script is finally properly multithreaded and generates all the data in less than a hour.

88

Database updated.

Comments on categories:

  • change 3._applications -> applications
  • change 9._freifunk -> freifunk
  • why do we have boot + boot_loaders? Can this be one category only?
  • why do we have devel + development? Can this be one category only?
  • dial-in:up + web_servers:proxies -> please no : in categories (no : enables linking to pages named like the category)
  • why do we have lang + languages? Can this be one category only?
  • why do we have lib + libraries + libs? Can this be one category only?
  • why do we have net + network + networking + network_support? Can this be one category only?
  • why do we have routing + routing_and_redirection? Can this be one category only?
  • why do we have utils + utilities? Can this be one category only?
89

Those are specifically LuCI add-on apps. Those are apparently the few LuCI apps that are hosted outside the main LuCI feed repo, e.g. in packages or routing repos,

  • They add new features to LuCI, so I would then prefer "LuCI_apps" category.
  • Or alternatively, the otherwise 100% identical luci apps "luci-app-xxx..." that are hosted the the LuCI repo have only the "luci" category, so the "3._applications" could also be just "luci" to match the other LuCI apps.

additional comment:

  • You might delete the "luci-i18n-YY-XX" apps. There are several hundred "luci-i18n-YY-XX" packages, one for each translation of each Luci app. I don't think that listing all those is necessary, as they are not independent apps. (and I do not think that they are installed seprately, as they get pulled in according to the language settings in connection of installing the main LuCI app, if I remember right)
    • Or alternatively, they should be something like "luci_translation"
90

Please consider that I'm not reviewing manually all 4000 packages and I rely on the wiki to find wrong stuff at each upload. Since I finally optimized the script to not take days, I can now focus on getting the data right.

change 3._applications -> applications
change 9._freifunk -> freifunk

yeah, these are names taken directly from menuconfig CLI interface, need to add a filter for that.

They are found under a parent menu about Luci stuff, so both are getting renamed into luci_something, as hnyman also asked

why do we have boot + boot_loaders? Can this be one category only?
why do we have net + network + networking + network_support? Can this be one category only?
why do we have routing + routing_and_redirection? Can this be one category only?

"boot" category is for boot loaders and is in "base" repository, "boot_loaders" is a sub category of "utilities" category contains the boot loader tools. Similar situation for the others.

Here the issue is that in the table these are just keywords with no hierarchy, so the names I read from the makefiles (that are used to generate a menu with submenus) in make menuconfig CLI don't make much sense.

What is the best way to deal with this?

I could add filters and stuff to rename categories, or make 3 dedicated fields: repository, category, subcategory.

Both have their pro and cons, so I'm asking some feedback here.

why do we have devel + development? Can this be one category only?
why do we have lang + languages? Can this be one category only?
why do we have utils + utilities? Can this be one category only?

Yes, it looks like the script failed to read the Category of Valgrind and its children packages, it should have been in "development" too, will have to see why and fix it.
Same for some other packages sharing the same makefile.

In general, if there are truncated names like "devel, net, utils, lang" it usually means the script couldn't read the package's category from the makefiles so it defaulted to what could read from opkg package lists (and there you find only section names which are truncated)
I didn't add a rename filter as I need to see the mistakes to make sure the script is reading makefiles correctly, and fix it if it isn't.
(there are like various different ways to add categories/submenu in packages, I likely have missed some of the ways)

dial-in:up + web_servers:proxies -> please no : in categories

Ok, adding a rename filter to change any : in the original name into something else.

91

[quote="hnyman, post:89, topic:123, full:true"]

  • You might delete the "luci-i18n-YY-XX" apps. There are several hundred "luci-i18n-YY-XX" packages, one for each translation of each Luci app. I don't think that listing all those is necessary, as they are not independent apps. [/quote]Yeah, I'll need to look at some details about them then I'll post some proposals for dumping most of that shovelware from the list without actually losing information.
92

Here the issue is that in the table these are just keywords with no hierarchy, so the names I read from the makefiles (that are used to generate a menu with submenus) in make menuconfig CLI don't make much sense.

What is the best way to deal with this?

I could add filters and stuff to rename categories, or make 3 dedicated fields: repository, category, subcategory.

Both have their pro and cons, so I'm asking some feedback here.

I am suspicious of any solution that assumes a fixed number of levels to a
hierarchy.

I suspect that a better approach would be to make the category reflect the full
hierarchy in one field ("base -> kernel -> netfilter") and then we can have the
single list make sense as well as creating links that reflect the full hierarchy.

David Lang

93

[quote="dlang, post:92, topic:123, full:true"]I am suspicious of any solution that assumes a fixed number of levels to a
hierarchy.

I suspect that a better approach would be to make the category reflect the full
hierarchy in one field ("base -> kernel -> netfilter") and then we can have the
single list make sense as well as creating links that reflect the full hierarchy.

David Lang
[/quote]I like the idea of showing "full hierarchy in one field" a lot, thanks.

Clarification: I proposed a solution with 3 levels of hierarcy as the hierarchy in makefiles can't go past 2 elements as there is only "category" and "submenu", so yeah we can't go beyond 3 elements total if I add "repository" on top.

94

Clarification: I proposed a solution with 3 levels of hierarcy as the
hierarchy in makefiles can't go past 2 elements as there is only "category"
and "submenu", so yeah we can't go beyond 3 elements total if I add
"repository" on top.

are you sure about this? I am pretty sure I've seen kconfig things that go more
than two levels deep (both in openwrt and in the kernel)

David Lang

95

Well, kinda, the script loads only 3 elements for that table field, and each of these elements ALWAYS has a single title (one or more words but a single title).

In the menuconfig you can find some packages in a submenu opened by a parent package.
Like for example coreutils, but afaik all these tools are in the same makefile, sharing the same category/submenu as the parent, so they go in the same category/submenu.

Again, this is my own understanding, by no means 100% correct. I learn as I go. Point out specific examples so I can look at them in the source too.

.

.

Btw, I still need to add kernel packages (kmod-*), I know.
These are missing from package lists entirely (I presume that Opkg generates the location based on some hard-coded data or something like that, don't know), so this part has to be dealt with a dedicated step in the script that I'm still writing. Most stuff is in makefiles in source/package/kernel/linux/modules/ but there are also some makefiles called modules.mk in /source/target/arch_name/modules.mk for device-specific drivers.

EDIT: no nevermind, found the package lists for kmod-* packages, it's in a folder near the flashable images.

96

Just noticed: Some empty strings in the description.

"This package provides support for '' in Asterisk11."

https://wiki.lede-project.org/packages/start?dataflt[Description*~]=package+provides+support+for+''

97

Can we live without the "File name"? I mean, it's a trivial scheme, and we're carrying around 4160 times "< version>_< architecture>.ipk".

98

Categories_pkg-categorys : community-packages, base_system # CTRL+Click for multiselect

There are 2 spaces between the last category and the '#' (this dmn forum software thinks that removing spaces from posts is a good thing, even if marked as preformatted text, where it should leave everything as is)
Can I have one space here, please?

Same (2 spaces) for
Maintainer_pkg-maintainer : | Etienne CHAMPETIER # Maintainer name

Edit: Just noticed that the double space is not always there. Seems to be dependant on the category. -> uniform appearance with only one space is needed to make automation work.

99

@bobafetthotmail

Since the dataentries will be read-only, we do not need this any more.

In general: We don't need any comments (=everything after the #) due to the read-only status of the dataentries. But of course we can also leave them in, should someone feel the need to read the wikisource of the dataentries.

100

@all

I got the automatic indexing of dataentry pages working now:
https://wiki.lede-project.org/packages/index/

This provides an alternative to the datatable on https://wiki.lede-project.org/packages/.
It is basic (only pagename of the dataentry is shown), but it can serve as a quick overview of all packages in a category, e.g. https://wiki.lede-project.org/packages/index/community-packages

6sec and you have 2255 packages on your screen.

Is this useful?
Please let me know your thoughts.

1 Like