Managing your 404s

There are two sides to 404 management: Managing the visitor experience and managing potential attacks. 'Luckily' Populate stores all requests made to a site that can't be matched to a resource and store these so that the site administrators can manage them.

Managing the visitor experience

There is nothing more frustrating than following a link to a 404 page, everyone's first response is back. As a site owner this is painful, you've generating traffic and lost it, I would suspect that most site owners will never be aware this has happened. Populate manages its URLs very carefully, nowhere do you ever need to type a URL in you always pick from a list and URLs are built dynamically. You can change URLs but the old URLs are automatically added to ouor 301 engine, so you should never really be able to create broken links.

Sometimes however we bring in sites from other platforms and sometimes pages or products need to be removed and links are still floating around in search engine indexes, social eaid post, bookmarks or in emails etc, so we need to map the old to the new and this is where our 404 report comes in handy. You can see what requests have been made and map them to a replacement.

Managing the vulnerabilities

Somewhat more worrying are the requests for resources that have never and will never exist on our platform. The vast majority of these request are wordpress related.

As an example, there is a file named xmlrpc.php that exists in older version of WordPress that has known security vulnerabilities, and it is suggested to remove it. It allows remote systems to update posts. Looking at one particular site that we manage there were 266 requests for that file from 200 different IP addresses in the last 30 days.

It's hard to know what or who is making these requests and what the purpose is. Is it to try and find a vulnerability in order to exploit it, or is it just a service that is checking to find out what tech we are using in order to market to us. There are two problems with Wordpress here

• Its been developed over a long period of time with a vast number of contributors of very deferring skill levels especially if we are talking about the plugins
• You can download and install it, pick it apart in order to find the vulnerabilities in order to exploit them on other websites.

Below is the top 25 requested files across our sites in descending order, none of which does or could exist in our platform

1. /xmlrpc.php
2. /wp-login.php
3. /mobile/javascript/config.js
4. /ads.txt
5. /about.php?520
6. /mobile/javascript/search_config.js
7. /mobile/javascript/bookmark_config.js
8. /sito/wp-includes/wlwmanifest.xml
9. /cms/wp-includes/wlwmanifest.xml
10. /site/wp-includes/wlwmanifest.xml
11. /wp2/wp-includes/wlwmanifest.xml
12. /test/wp-includes/wlwmanifest.xml
13. /wp1/wp-includes/wlwmanifest.xml
14. /news/wp-includes/wlwmanifest.xml
15. /wp/wp-includes/wlwmanifest.xml
16. /website/wp-includes/wlwmanifest.xml
17. /wordpress/wp-includes/wlwmanifest.xml
18. /web/wp-includes/wlwmanifest.xml
19. /blog/wp-includes/wlwmanifest.xml
20. /xmlrpc.php?rsd
21. /wp-includes/wlwmanifest.xml
22. /.env
23. /.git/config
24. /admin.php
25. /shop/wp-includes/wlwmanifest.xml