Popular Bible website blocked from results recently

Within the past couple of months, searches like “John 5” or “Matthew 6” no longer have biblegateway.com in the results list. Even searches like “biblegateway john” do not give a single result from biblegateway. It works perfectly fine with google’s search engine and Bing. I used to use it all the time for the past couple of years and now suddenly it’s broken. So what is going on? What did the brave indexing bot do to break it?

3 Likes

@Ph0t0n

Strange, indeed. Using a Brave Browser New Private Window, went to:

https://www.biblegateway.com/

No trouble reaching that website . . . BUT:

Used Brave Search. Entered as search criteria in the search field (not in the URL address field); used search operator site:biblegateway.com:

  • john 5 site:biblegateway.com

Results, NOTHING:

NOTE: “Search operators were not applied” because “Too few matches were found”


Searching via DuckDuckGo, no trouble with results:


Yeah it’s really weird. Even just searching for “biblegateway” doesn’t give a single result from biblegateway.com. Could it be that biblegateway intentionally blocked brave’s indexer but allows duckduckgo, bing, google, yahoo, etc to index it? I don’t understand what the point of that would be. But then again, I can’t think of why Brave would intentionally block biblegateway but not other similar websites like bible.com. My best guess is that it’s a technical issue, but even that seems strange because I’m 100% positive that just a few months ago it worked perfectly.

Very mysterious indeed…

1 Like

@Ph0t0n

What is the Web Discovery Project?

https://support.brave.com/hc/en-us/articles/4409406835469-What-is-the-Web-Discovery-Project

EXCERPT:

The Web Discovery Project allows you to contribute anonymous, generalized data.

The Web Discovery Project is designed to prevent us from associating this data with you. This means there’s no data for Brave to sell to advertisers, or lose to theft or hacking, allowing us to promise through technology rather than words.

Brave’s Web Discovery Project is opt-in only, and totally transparent.

I’m not sure why you sent this article on the Web Discovery Project and how it relates to the topic of this post. Are you suggesting that if I enable it and search for biblegateway it will start indexing the site or something?

@Ph0t0n

You want to improve the chances for biblegateway.com showing in Brave Search results . . . then you might join the Web Discovery Project.

From the Brave Support - Brave Help Center article, first paragraph:

"If you opt in, you’ll contribute some anonymous data about searches and web page visits made within the Brave Browser (including pages arrived at via some, but not all, other search engines).

This data helps build the Brave Search independent index, and ensure we show relevant results to your search queries and support more relevant experiences with Brave products and services.

And further down in that article:

Can I opt-out of the Web Discovery Project, even after I opt-in?

By default, all users are opted out of the Web Discovery Project. If you’ve chosen to opt in , you can opt out again at any time.

Whatever you choose—opt in or opt out—your experience in Brave or Brave Search will not change.

To opt out, open a new tab in the Brave browser and click Settings . Scroll to “Web Discovery Project,” and toggle this setting off.

Agreed. I use biblegateway.com almost exclusively, but it never shows up on Brave searches.

Are you saying that every other bible site that shows up on Brave searches has been enabled because of the Web Discovery Project? I am dubious. Something is preventing biblegateway.com from showing up. It seems to me that Brave crawling/indexing it might help.

@Mattches

https://www.biblegateway.com seems to be blocked or ranked so low as to be in exile from Brave Search results.

If I search, using the Brave Search field, only for: "https://www.biblegateway.com" the website is not in anyof the few results.

If I search repeatedly, using the Brave Search field, for Jacob "https://www.biblegateway.com" there is eventually 1 result.

If I search, using the Brave Search field, for Jacob site:biblegateway.com there are no results.

Forwarding this to the Search team – will reply back here when I have more information.

2 Likes

Just to add to this, I am finding immense frustration with the same "Search operators were not applied. Too few matches were found” problem, resulting in me ditching Brave Search entirely unfortunately until this is fixed.

The entire point of search operators is to narrow down your results, EVEN if that means very few results, even less than 5, or even 0. All of those cases are useful.

That is literally the whole point of using them. If I am trying to narrow down my search specifically, why would I want Brave Search to completely ignore my specific search operators and instead give me endless irrelevant results? I just don’t understand how this conclusion was reached. If being specific to narrow results found very few matches, it makes no sense to go in the complete opposite direction and ignore my specificity, instead of showing those few matches because that was the reason for being specific which naturally would find fewer matches…

There is nothing wrong with showing 0 results or very few results if a user is using search operators, because that is exactly what we’re looking for. If [search term] doesn’t exist online, perfect, thanks for letting me know. If [search term] exists only on 1 website, great, that’s highly likely to be what I’m looking for. The purpose of them is to know whether there ARE results for this, to find SPECIFIC results, to REFINE results, all for countless different scenarios. Brave Search is simply unusable for both my personal and work use by it choosing to ignore search operators.

It makes far more sense to show a link that says “Would you like to ignore search operators to see more results?” And if you click that, THEN it shows endless results that ignore them.

TL;DR: Brave Search should not assume what users want to see and ignore search operators at any point in time ever, because the narrowing of results is the entire reason they’re used, even if this means very few or no results. Instead, please show the few/no results, THEN show a link prompting the user to CHOOSE to ignore search operators, or simply show those results underneath clearly separated from the results we specifically searched for. Thank you.

P.S. Here’s a great example: I searched “Search operators were not applied” in Brave Search to find anyone previously talking about this, and Brave Search once again ignored my search operators due to “too few matches”, so I ironically had to use Google which immediately identified this thread.