Update again, I found the culprit: varnish cache on the server level was causing the issue. I excluded the pages from being cached and it solved the problem.
However, it used to work with varnish enabled, was something changed in the latest updates? Any solution to have the URL feature work with varnish enabled?