Hosts: [REQ] Add option to remove comment comment lines from source lists

[kronflux]

I see there's the option to remove comments on the same lines as entries, but I don't see an option to leave out comments entirely.

There's a lot of whitespace and a lot of comments throughout. I'd love to be able to clean it up, and leave only the entries.
It's visually unappealing, but more importantly it contributes to filesize, which in the case of routers and similar devices can sometimes be very limited. The more space we can save, the more bad hosts we can block in the future!

It makes sense, of course, to have a line at the beginning of each list, such as "# Start of LIST X - Updated on Xtober XX 20XX" for credit, informational and organizational purposes, but further comments or whitespace are unnecessary, and are in inconsistent formatting due to being created/maintained by different people/groups.

Hopefully this is appealing to more than just me, but either way thanks in advance for considering the request!

Cheers!

[Laicure]

+1 for the above proposal, a -min (minimal) parameter will do.

Something like below:

[kronflux]

maybe a --clean [min]

if [--clean] specified, remove comments and whitespace, add list credit and updated date comments

if [--clean min] specified, remove comments and whitespace, add no comments at all, perform alphabetical sort on list
(like shown in @Laicure's example)

[kronflux]

Examples:

--clean:

---

--clean min:

The current default hosts file in this repo is 1.15 MB. The minimum reduced example I posted is 1.03 MB.
On very low-end routing or firewall devices, 0.12 MB can actually make a big difference.

[ScriptTiger]

Depending on your uses, my scripts can automatically do this just by adding the comment characters to the ignore list (the syntax will vary depending on if it's regex or literal), and they already automatically clean white space. Just keep in mind the reason no one distributes such lists is that the distribution of uncredited files would be in violation of the respective licenses. So needless to say, this would be for personal use only.

[ScriptTiger]

This actually brings up an interesting segue for a network automation script I have for updating network devices automatically. I haven't made it public yet, but I have been curious as to the level of interest there might be for it.

[kronflux]

yeah the intent for removing the credits would be for low-end devices only, where space or memory are extremely minimal(or in some cases, the device may simply not be able to process a large hosts file). The default(and preferred) option would be to leave credits in place.

In reply to your second comment, automation is gold in IT! Even if it wasn't something people ended up using, at the very least people can learn from it. So definitely make it public! :) :+1:

[ScriptTiger]

I'll work on a public version in the coming weeks. I will probably only officially support SSH/TELENT out of the box to start. I also have a version that does support automation through HTTP/HTTPS POSTing so you can make multiple changes with a single command to the network device and dramatically speeds things up, however I will need a lot more time to figure out how to make something that would be generic for anyone to use, as HTTP/HTTPS form data is not in a linear structure and is wildly different from device to device.

[StevenBlack]

Hi everyone, @kronflux @Laicure @ScriptTiger

I don't think this can happen. Here's why: It's all about licensing.

Simply put, some of our sources have licenses that require copyright and licensing notices to remain intact.

[kronflux]

Would that matter if it's not actually released in that format?
I would imagine keeping them in tact on github, and adding the option to use for personal cases should be perfectly fine, since it's the user's own use case, not something that's public.

It would be the same as me manually removing them in a text editor.
I'm not violating anything, because I'm not releasing anything anywhere.

I get where you're coming from, but since it would only be an option, which only runs in a manual use-case scenario, I don't see a problem.

[StevenBlack]

@kronflux good point. OK.

[ScriptTiger]

Yeah, it's all about the intent to distribute, as they say in the hood.