The LiteSpeed Crawler is a part of LiteSpeed’s WordPress cache plugin. Usually, a cache page is created once the first user visits an un-cached page, any other user that will visit the same page will see the cached page.
Enabling the crawler functionality will simulate “visitors” crawling around your website pages, making sure your website cache is up to date and generated constantly.
Here are the steps needed to enable the cache –
- Enable crawler in LiteSpeed server settings
- Making sure relevant php functions are enabled
- Install any missing php module
- Extra step for cPanel servers
- Setup crawler in LiteSpeed WordPress plugin
Enable crawler in LiteSpeed server settings
Log into your LiteSpeed’s server admin console, the default port for that is 7080 (https://SERVER_NAME:7080), then navigate to –
Configuration -> Server -> Cache
On that page, edit the “Cache Storage Settings“: Select “On“, “Crawler“, “ESI“, and then Save and restart LiteSpeed
Making sure relevant php functions are enabled
Edit the relevant php.ini file or all of them, they are usually located in “/usr/local/lsws/lsphpXX/etc/php.ini“, that really depends on your setup.
Look for the “disable_functions” section, and make sure that the following functions are *not* listed there –
curl_multi_exec, exec, pcntl_exec, popen, proc_close, proc_open
After editing and saving the file, restart LiteSpeed.
LiteSpeed crawler is using the “PHP XML” extension, make sure it’s installed and activated –
(XX stands for the version, for example, lsphp70)
yum install lsphpXX-xml
Extra step for cPanel servers
Log into your cPanel server, edit the “pre_main_global.conf” file, and add the following line –
CacheEngine on crawler
To apply change changes, execute the following –
Restart LiteSpeed once again.
Setup crawler in LiteSpeed WordPress plugin
- Navigate to LiteSpeed cache plugin settings
- Go to crawler
- Select General settings and enable crawler, then set server load and threads as appropriate for the server.
- Save settings