Will the new removeparam feature work with custom filter lists

I have been using removeparam feature inside UBO for quite some time. I use it via subscribing to a custom filter list.

Brave added this feature in their brave adblock rust just now.
So will my custom filter list work 100% just like it did on UBO.
If it does, there will be no need for me to use UBO. (thus reducing resource usage)

https://github.com/brave/brave-browser/issues/27744 (at bottom you will see it)
https://github.com/brave/brave-browser/issues/23927

1 Like

It already works, any feature works with Custom filters/rules, the only exception was/is $badfilter but it was because of different Adblock engines in Brave.

They did a merge of the adblock engines, I noticed the $badfilter doesn’t work for default lists, but now they work for custom lists, so now if I load the same default lists as custom lists (and remove the default lists) I get the same features as default lists.

There are some differences in the way you write the syntax though, and some behaviors that seemed strange the first day it was available in Nightly. If you want me to explain them I can, you probably have to edit some rules to be compatible with Brave, but answering your question, so yes and no, they will work, some, but not all for different reasons.

It can also cause crashes, so you have to be careful with it. I have to create yet another Github account to report the crashes and see if they are meant to be for the way it was implemented or they can be fixed…
Which makes me wonder why they are pushing it to stable, of course, in the pull they test it with the ‘simple’ removeparam rule, but they should have tested more uses for it, especially when @fmarier in https://github.com/brave/brave-core/pull/15943 linked this list from Adguard https://github.com/AdguardTeam/FiltersRegistry/blob/master/filters/filter_17_TrackParam/filter.txt and he got kind of ignored with his question.

But it seems to work with that set of lists but there are many issues that nobody reported and I was lazy to create a new Github account to do that, but I guess I have to.

It should generally work the same way. The only two known limitations at the moment are:

We’ll work on implementing these when possible, but note that they’re much less common than other removeparam rules.

There are some differences in the way you write the syntax though, and some behaviors that seemed strange the first day it was available in Nightly. If you want me to explain them I can, you probably have to edit some rules to be compatible with Brave, but answering your question, so yes and no, they will work, some, but not all for different reasons.

That’s unexpected, I’d appreciate if you could share more about the differences you’ve noticed!

It can also cause crashes, so you have to be careful with it. I have to create yet another Github account to report the crashes and see if they are meant to be for the way it was implemented or they can be fixed…

This too; it’d be great if you could provide some more detail about the crashes. GitHub issues would be awesome but here is fine too if you’d prefer!

So, all of these (the UBO screenshot) are related to URL’s. Others I have manually disabled them and added each one by one into brave shields manually. Brave already had default UBO lists and some extra regional lists in brave://settings/shields/filters some of which I have turned ON.
Some were missing in the brave shields database like AdGuard annoyances https://filterlists.com/lists/adguard-annoyances-filter-ublock-origin so I added them manually.

But, I left out adding URL lists because I have no idea if they work properly in Brave shields (brave adblock rust). As far as I know, Peter Lowe list is included in brave by default, so will Dan Pollock’s list also work in brave the same as UBO. Also, will adguard URL tracking protection list work in brave just like Peter Lowe list.

On other note, brave should make a FAQ or support artcile on official support webpage to confirm that which ‘popular’ lists actually work and which do not. If they work, how much percentage do they work compared to UBO.

Most rules work in Brave.

The only ones that don’t work are Procedural Cosmetics as anyone knows, you know, the ones with :has, :has-text, :upward which are the 3 most used Procedural Cosmetics.

Then specific uBlock rules like $denyallow, $all, $popup

Then some specific ones aren’t supported because of performance reasons, like generic exceptions, the ones that are like #@#example, so what Brave team does is to manually add one by one the exception for each domain or $ghide which will remove all generic cosmetics in a page.

Positive and negative lookahead in regex rules don’t work, which are the ones that have ?! or ?=
so some uBlock rules will not work because of that.

There is no support for entity in $domain so if a rule has $domain=example.* it won’t work.

There is also no !#include support yet which is to include several lists in one, like uBlock filters do, Brave has to add individually all lists that are created by year, for example they are already writing rules to 2023 one. uBlock only needs to use !#include and it will be loaded automatically with it.

there are other rules but not used really much to really worry about them.

Some rules can be workarounded by using CSP or scriptlet injections, like denyallow can be done with CSP, popup can be done with CSP but there is also a scriptlet called window-close-if, which can do the closing like popup does, the problem is Brave uses uBlock lists so they should add support for popup so the rules are compatible.

But as you can see not much has to be supported, Procedural Cosmetics is a big world to support anyway, with so many Procedurals that are available, so in that Area it is still a lot of to do but not in Network filters.

About adguard URL tracking protection list, well, $removeparam, the thing is if you take ||example.com^$removeparam=whatever it will work, and that syntax is used by lists.
but for some reason anything else needs $document,removeparam=whatever to work, so most rules will not work if they are generic or are anything but ||domain^.
Of course, editing the file and add the $document takes few seconds, but you have to manually do it.
So I reported it because it’s important to bring compatibility, so lists just work, but for now manually editing lists is the only way of getting almost full removeparam support (excluding the ones with regex on them)

Edit: Crashes were fixed already by @antonok already, it works great now! so thanks to him for that.

Edit2: @chh_68 the fixed are being pushed quickly to stable as well, since it can hard crash in Brave.
Removeparm works great, so once the changes are pushed (to avoid crashes), all you need to do (continuing with what I said about compatibility) it’s to load the list Adguard removeparam list locally, and all you do is find $removeparam and replace it for $document,removeparam which even Notepad can do in 1 second. This is the only way to make those rules work, and you can easily update it from time to time, select all, paste, find, replace and save.

2 Likes

Hey emi, I wanna ask if i subscribe URL Shortener filter Will it works on Brave Browser?

Which list do you want to add?
I better ask to be sure, but if if you refer to removeparam one or one with rules with it.

Brave team/=Antonok already fixed the main issues that caused incompatibility and crashes, there is still a little tiny bug, but I have the workaround for that.

But now anything like this works:

||example.com^$removeparam=test :white_check_mark:

example.*$removeparam=test :white_check_mark:

example.com/test/*/search$removeparam=test :white_check_mark:

$removeparam=test :white_check_mark: but sometimes :x:

Most rules work, but some parameters won’t get removed. The workaround for these generic removeparam rules is to add || at the beginning of the rule.

With a decent text editor like VSCode with regex should be easy find ^(\$removeparam) and replace for either ||$1 (if supported) or ||$removeparam

if they look ||$removeparam=test for some reason they will work 100%, but it will be fixed eventually, of course.

And then, anything else is not supported yet, like: ||example.com^$removeparam=/fx_(source|medium|campaign)/ :x: (regex support - planned)

@@||example.com/*&test=s$removeparam=test:x: (exceptions support - planned)

||example.com^$removeparam :x:

?uclick=*&uclickhash=$removeparam :x:

This filter https://raw.githubusercontent.com/DandelionSprout/adfilt/master/LegitimateURLShortener.txt

Late reply, as I busy for a week.

I just tested it both on stable and nightly and it seems to be 50% similar to UBO/Adguard removeparam.

For eg., If you take some scenarios/examples from https://kb.adguard.com/en/general/how-to-create-your-own-ad-filters#removeparam, like http://google.com/page?utm_source=s&utm_referrer=fb.com&utm_content=img, it should turn it into http://google.com/page but in brave it only removes first part while second part just stays https://google.com/page?utm_source=s&utm_content=img.

In brave, it also does not http://example.com/page?param=1&another=2 it at all, since it should turn it to http://example.com/page?another=2.

Tested with Adguard URL list and Dandelion Sprout URL shortner list.

Emi has written some stuff, I have no idea what it is. So, I will continue using removeparam feature by UBO until brave shields is 100% similar to UBO.