Meh, I can't get it to log the change without using dwpage.php, nor to delete packages without logging it.
So I'm doing a second run with dwpage.php after the indexer.php to log the change. Since the indexer has loaded all things already and the wiki is working for people looking at it, it's not an issue if the second pass (that is useful only for logging) is slow.
All these things are run as child processes of the script so they inherit the low priority I talked above, so again should have no impact on server performance.
I also added a comment to all written files as requested, so people will not be upset if they find they can't edit them or that their stuff isn't there (yet).
I'm running a rescan now with the new script, this evening should have finished.
I noticed that the indexpages always get recreated, e.g. https://lede-project.org/packages/index/network---filesystem, even if no changes happen to them. If you don't change anything on the pagequery command, these indexpages do not need to be updated.
Can you change the update script so that it only updates dataentries / indexpages which really have changes?
[quote="tmomas, post:165, topic:123, full:true"]I noticed that the indexpages always get recreated,[/quote]All package entries are recreated on each run like that.
To fix that I will need to add some logic to detect if the current package entry was changed. I'll see what I can do.
[quote]And btw... I noticed that we now get two entries in the changelog: deleted + updated, see e.g. https://lede-project.org/packages/pkgdata/kmod-rtl8192de?do=revisions
Can this be reduced to one?[/quote]Yeah, if I add the logic to detect what was changed I can reduce that to a single "updated", which will happen only if the page needs to be changed.
[quote="tmomas, post:167, topic:123, full:true"]OK, for the package dataentries, but what about the index-/pagequery pages, e.g. https://lede-project.org/packages/index/network---filesystem?[/quote]Same thing. There is no logic to detect what was changed and the indexes can change too, so the script is simply erasing and then rewriting both indexes and packages.
As I said, I'll have a look into this, for the sake of reducing logspam.
Ok, now it shows only "update" in the logs, it seems the check I added is thinks the file is different even when it is not. Will have another go this weekend.
Will switch it to track LEDE release (so the table will have only packages in LEDE 17.01) as that's probably the most important target, but it's supposed to eventually track all LEDE releases and generate different tables for each LEDE release (as package versions and other info may differ).
Thanks for your quick reply. From here, I think that the most important thing to do would be to document those limitations on the table's page, so readers can be aware of them.
Yes, the switch to LEDE release didn't work correctly the first time.
I've already fixed it and launched a rescan before you posted. The table and indexes should be back again by tomorrow.
I'm seriously considering to clone with rsync the LEDE server in a Virtualbox VM on my PC so I can test things I can only test on the server (wiki interactions, mostly) without screwing up the wiki.
rsyncing: Good idea!
There was once the idea to have a demowiki for doing tryouts before going productive, but I think this got lost back then when the wiki was created... :-/
I have my own demowiki on a rpi2, but that can not really be compared to a full clone of the wiki on the same / similar server.
What was the reason to do this our self and not use some existing software from any Linux distro?
I would love to see some functionality seen in the Archlinux package data base also implemented here.
One of the things I like the most is the package "overview" site that shows a couple of nice things it would be too much to describe, for an example look at the site from the systemd package.
Another nice function is the flag out-of-date which can be used to signal that there is a new release from the upstream developers.
[quote="Wabuo, post:177, topic:123, full:true"]What was the reason to do this our self and not use some existing software from any Linux distro?[/quote]Because I don't know them, nor how to use any such existing software. The hard part was the datamining script, I have no issues in changing the way it writes down data so it can use a different frontend.
[quote]I would love to see some functionality seen in the Archlinux package data base also implemented here.
One of the things I like the most is the package "overview" site that shows a couple of nice things it would be too much to describe, for an example look at the site from the systemd package.[/quote]
There is a partially similar package page, like for example here https://lede-project.org/packages/pkgdata/wireless-tools
But adding the "dependencies", "required by", "package contents" boxes is probably too much for the wiki software we are using here.
