We have an app that is communicating with WooCommerce via HTTPS
During development on Windows we were getting “HTTP/1.1 403 Forbidden” and came across this
The solution presented there, setting UserAgent, worked, without exactly explaining in any detail as to why
‘Mozilla/5.0 (Windows NT 6.1; WOW64; rv:12.0) Gecko/20100101 Firefox/12.0’ was a correct and valid value (over the default of ‘Mozilla/3.0 (compatible; Indy Library)’ ).
We now have three cases in our system with “HTTP/1.1 403 Forbidden” and it seems they are all in large commercial environments and that their high level virusscanner/firewall is blocking our request.
This is because one company has conveyed to us that the message they are seeing is
"User Agent “FireFox 12.0(Mozilla/5.0 (Windows NT 6.1; WOW64; rv:12.0) Gecko/20100101 Firefox/12.0)” and
"Not allowed to use this browser”
They are using a product from ZScaler, which is ‘doing’ this, but I am not getting much of a useful response from ZScaler.
Can someone explain, is this because the useragent is somehow identifying an old/buggy browser that should be blocked from working?
If, so what should UserAgent be properly set to, as it seems Windows ‘requires’ a browser be set, yet it seems any setting might eventually cause 403 again when any particular browser setting is discovered to have issues?
I’ve seen this many times before - not just with Indy but with other http client libraries. Sometimes, just changing it from the library default is enough, sometimes you have to mimick an actual browser.
The UserAgent header is meant to be used by servers to do things like handle the differences between platforms and browser capabilities etc - however many firewalls use this to block would be hackers, so if someone attempted hacking in the past using the default Indy Useragent, then that would make it’s way into firewall defences at some point.
So in short, what you did is the correct way to deal with this issue.
A Bit more research shows that FireFox 12 was released in 2012
The stackoverflow post was made in 2014 when FireFox 28 was current
FireFox 112 was released 11/4/23
So, I guess you guys are saying I/we/us need to set useragent to some recent browser combination that is unlikely to be blocked by any organization’s firewall, due to bugs, or just plain whimsy, and to periodically update it to be current/recent?
Yet another thing broken on the internet that seems like it will never be fixed.
It would at least be worth trying.
Although web stuff is probably pretty permissive, it would make sense if might get cranky at very old settings.
(Sidenote : My previous Samsung phone is used by my 6yo to watch YouTube Kids … and after running out of battery, it wasn’t working. It was connected to our wifi … but then I noted the system date was wrong by 4 or 5 years, and that was upsetting YouTube until I changed it to the current date/time. )
Our original validation just used ‘standard rest’ via https, and we never had a single 403 forbidden (with the default useragent)
Now that our website is Wordpress, ‘The system that powers 43% of the internet’ and uses woocommerce, we had to change the useragent just to get out of windows and currently have 4 different corporate clients who are blocking at their firewall.