For cPanel & WHM 11.46
Often, when users compare bandwidth usage statistics that were provided by log analysis programs such as AWStats, Analog, Logaholic, and Webalizer to bandwidth statistics that were provided by cPanel & WHM, they are surprised to find apparent discrepancies.
These apparent discrepancies arise as a result of the way different programs measure bandwidth.
The purpose of this document is to explain the differences in measurement methods. Our goal is to help web hosts and website owners understand how to obtain the most accurate bandwidth report.
Where to find bandwidth data in cPanel & WHM
Bandwidth data in cPanel & WHM comes from two sources: data stored in summary files and the data stored in RRDtool databases.
Data stored in summary files
Several cPanel & WHM interfaces display (mainly numeric) bandwidth usage information stored in summary files. These interfaces include:
- WHM >> Account Information >> View Bandwidth Usage.
- WHM >> Account Functions >> Limit Bandwidth Usage.
- cPanel >> Stats >> Monthly Bandwidth Transfer.
- cPanel >> Logs >> Bandwidth >> monthly bandwidth usage pie charts.
- cPanel >> Logs >> Bandwidth >> select the domain >> Bandwidth By Day.
All of the information on these pages is generated by the same bandwidth monitoring tool.
In contrast, AWStats, Analog, Logaholic, and Webalizer use a different method to process bandwidth. This is why it sometimes appears that there is no correlation between the data shown on the bandwidth usage information interfaces and the data shown by AWStats, Analog, Logaholic, and Webalizer.
Data stored in RRDtool databases
Most bandwidth graphs are generated from usage information stored in RRDtool databases. These graphs include:
- cPanel >> Logs >> Bandwidth >> Traffic for Today graph.
- cPanel >> Logs >> Bandwidth >> Traffic for Last 7 Days graph.
- cPanel >> Logs >> Bandwidth >> Traffic for Last 12 Months graph.
Data for these bandwidth graphs are calculated with a different method from the summary file data above.
What causes the discrepancies?
Differences in measuring daily bandwidth statistics
The information presented by cPanel & WHM is likely to differ from the day's usage as shown by log processing programs, such as AWStats, Analog, Logaholic, or Webalizer. There are three possible reasons for this:
- The server administrator can set log processing and bandwidth processing cycles independently. This means that the two statistics can be generated hours apart, in which case, they may vary widely.
- Even if the server administrator attempts to synchronize log and bandwidth processing cycles, the system will always process the statistics at least eight minutes apart.
- The statistics are processed such that usage data from five minutes before the time at which it was processed, up to the time at which it was processed, is always held until the next processing cycle. Therefore, those statistics would appear in a later report.
In short, daily bandwidth usage numbers are far too volatile for the bandwidth and log processing results to match.
Overall differences in measuring bandwidth
There are further differences between the methods used by log programs and cPanel & WHM to process bandwidth usage.
How the log processing programs measure bandwidth
The log processing programs extract the amount of bytes transferred from the Apache combined access log. The Apache combined access log records incoming data requests. One of the pieces of data recorded is the size of the requested file, measured in bytes.
The size amount of the requested file:
- only shows the full size of the content of the file being requested. It does not reflect the actual number of bytes that were transferred.
- measures the content of the file only. This does not include the size of the headers of the response.
- does not include the size of the request.
In other words, the log processing programs measure bandwidth by the size of the data that was requested, not the amount the data that was actually transferred.
How cPanel & WHM measures bandwidth
To measure bandwidth, cPanel & WHM combines the amount of bytes of an incoming data request with the outgoing transferred data (the response to that request). cPanel & WHM obtains this data from the Apache logs.
The amount of bandwidth that is reported includes:
- The total bandwidth that was used by the incoming request that solicited the content.
- The total bandwidth that was used by the complete response: the sum of the size of the content plus the header.
How this affects the results
For most websites, these differences will be relatively small. The headers are a few hundred bytes; the request itself is usually much less. Given reasonably sized output content, this overhead works out to a few percentage points' difference.
There are a certain types of request/response combinations in which these differences begin to make a significant impact on the bandwidth measurement:
- Large numbers of non-success responses — Because redirects and errors have virtually no content, and are mostly composed of headers, the bandwidth reported by the log processing programs will be much smaller than that reported by cPanel & WHM.
- Large numbers of very small responses — For transferred content below one kilobyte, the log processing programs will show a smaller bandwidth usage amount than that reported by cPanel & WHM. This is because the combined size of the response headers plus the request can often be larger than one kilobyte.
- Large requests cancelled by the user — As stated above, to measure bandwidth, log processing programs show the size of the data requested, not the amount the data that was actually transferred. This means that these programs do not make a distinction between completed and cancelled requests. Therefore, a high number of requests that were cancelled by the user may cause the amount of bandwidth reported by cPanel & WHM to look substantially smaller than the bandwidth reported by the log processing program. This situation occurs often with sites that serve a large number of video or audio streaming requests.
- For example, 100 people request a 1GB video, but cancel the request after only 10% of the video has downloaded. cPanel & WHM would report that less than 10 GB of data were transferred. However, because of the way they obtain their information, the log processing programs would report that 100 GB were transferred.
Which report should I use?
You will want to use the more complete bandwidth measurement reported by cPanel & WHM, for example on the cPanel Bandwidth interface, or the WHM View Bandwidth Usage interface. Log analysis programs, while not as accurate, are useful for information about pageview statistics, referrals, and other information.
You should set bandwidth processing cycles as desired on the Server Configuration >> Statistics Software Configuration interface of WHM, in the Bandwidth Processing Frequency area of the Schedule Configuration interface.
This will allow you to balance server performance with latency reports and, given those constraints, will provide the most accurate bandwidth usage statistics possible.
For up-to-date usage information, you may process an individual user's statistics at any given moment. You can do this on the Server Configuration >> Statistics Software Configuration interface of WHM, under the Process Statistics for User heading.
- View Bandwidth Usage — Documentation for the View Bandwidth Usage feature in WHM.