Details
-
Type:
Task
-
Status: To-Do (View Workflow)
-
Priority:
Major
-
Resolution: Unresolved
-
Affects Version/s: None
-
Fix Version/s: None
-
Labels:None
-
Story Points:1.5
-
Epic Link:
-
Sprint:Spring 2, Spring 3, Spring 4, Spring 5, Spring 6, Spring 7, Spring 8, Spring 9, Spring 10
Description
Keeping track of the number of times that bioviz.org is accessed and IGB is downloaded after outreach opportunities would be a great way of measuring the impact that our outreach efforts are having. There may already be an xml document in place that contributes to this functionality within the website.
PK: comments it would be good to have in place before the JDK21 release because this would give us a baseline right at that important milestone
Attachments
Issue Links
- is blocked by
-
-
- Closed
-
Activity
Question:
- How do we turn off log rotation?
- If we collect all the data on a particular date, how can we empty out the existing logs (so that we don't double-count stuff?)
Request:
- Can we record the IP addresses of accesses and also let's tell Apache to report the domain they're coming from as well?
- Let's log updates.xml in its own log file
I've added the following code snippet to the configuration file so that any instance of IGB being opened will be logged to its own file (called usage.log):
SetEnvIf Request_URI "^/igb/releases/current/updates.xml$" usage
CustomLog ${APACHE_LOG_DIR}/usage.log combined env=usage
Both downloads.log and usage.log will record the IP address and the site that a user was on prior to either downloading an IGB installer or launching IGB, respectively.
I'm not seeing an option to disable log rotation for specific log files that doesn't involve deleting files which would disable log rotation for all files, so here's what I've configured instead:
- The access and error logs will continue to rotate as usual
- The newly created downloads and usage logs will rotate weekly and will be kept for a year before being deleted
This rotation should allow us to retrieve all of the downloads and usage log data at least once a year in order to conduct some sort of time series analysis where we can show the impact that outreach might be having on IGB's usage. This also ensures that logs aren't being added to indefinitely which would lead to memory issues.
You can view the file I edited to achieve these changes here: /etc/logrotate.d/apache2
Ann Loraine You should be able to deploy these changes now if I've addressed all of your questions and requests!
Working on modifying BioViz configuration playbooks to incorporate above changes.
I've added a configuration file to the log rotation directory (/etc/logrotate.d/) called "access" that manages the rotation schedule of downloads.log and usage.log. To check that it will rotate correctly, I ran the following code:
logrotate -d /etc/logrotate.d/
Dr. Loraine mentioned that she'd like to configure the host to put these two log files in their own subdirectory (i.e., apart from the other log files). If/when that happens, I can update the file path being used in this new configuration file I mentioned above to point to that subdirectory.
When IGB launches, it retrieves a file named "updates.xml" from the Bioviz.org Web site.
By counting the number of times this file is downloaded, we can track the number of times a user (anywhere in the world) has launched IGB.
IGB (so far as we know) is the only program that requests this file from our Web site.
This file is located here: https://bioviz.org/igb/releases/current/updates.xml
Our Web server used to deliver BioViz.org content (apache2) has log files that record requests to our site. The logging process for Apache2 is highly configurable. For example, it is possible to customize the logging format (e.g., the information getting logged) as well as how long the log files are kept on the server - called "rotation" schedule.
So, for this task, it would be smart to investigate the different logging options to see how we can best record IGB accesses.
In addition to tracking the "updates.xml" file, we ought to also track how often IGB installers gets downloaded.
For this task: