Uploaded image for project: 'IGB'
  1. IGB
  2. IGBF-2572

Configure InfluxDB to only store 2 weeks of data

        Details

        • Type: Task
        • Status: Closed (View Workflow)
        • Priority: Major
        • Resolution: Done
        • Affects Version/s: None
        • Fix Version/s: None
        • Labels:
          None

          Description

          Situation: The current implementation of the CyVerse Terrain API endpoint monitoring using Grafana/InfluxDB/Telegraf will continue to add additional data to the InfluxDB indefinitely.

          Task: Configure InfluxDB so that it only stores only the most recent 2 weeks of data.

            Attachments

              Issue Links

              relates to

              Task - A task that needs to be done. IGBF-2484 Investigate: Tooling to automate testing Terrain API

              • Major - Major loss of function.
              • Closed

                Activity

                Ascending order - Click to sort in descending order
                nfreese Nowlan Freese created issue -
                nfreese Nowlan Freese made changes -
                Field Original Value New Value
                Epic Link IGBF-2376 [ 18533 ]
                nfreese Nowlan Freese made changes -
                Link This issue relates to IGBF-2484 [ IGBF-2484 ]
                nfreese Nowlan Freese made changes -
                Assignee Chester Dias [ cdias1 ]
                ann.loraine Ann Loraine made changes -
                Sprint Fall 4 Oct 26 - Nov 6 [ 107 ] Fall 4 Oct 26 - Nov 6, Fall 5 Nov 9 - Nov 20 [ 107, 108 ]
                ann.loraine Ann Loraine made changes -
                Rank Ranked higher
                cdias1 Chester Dias (Inactive) made changes -
                Assignee Chester Dias [ cdias1 ]
                cdias1 Chester Dias (Inactive) made changes -
                Status To-Do [ 10305 ] In Progress [ 3 ]
                Hide
                Permalink
                cdias1 Chester Dias (Inactive) added a comment -

                CREATE RETENTION POLICY "Retention_policy" ON "Grafana" DURATION 2w REPLICATION 1

                Reference Link: https://docs.influxdata.com/influxdb/v1.8/guides/downsample_and_retain/

                Show
                cdias1 Chester Dias (Inactive) added a comment - CREATE RETENTION POLICY "Retention_policy" ON "Grafana" DURATION 2w REPLICATION 1 Reference Link: https://docs.influxdata.com/influxdb/v1.8/guides/downsample_and_retain/
                Hide
                Permalink
                cdias1 Chester Dias (Inactive) added a comment -

                Policies can be created via Grafana UI on
                https://grafana.bioviz.org/

                Login->Left panel menu ->explore and write the above query

                Show
                cdias1 Chester Dias (Inactive) added a comment - Policies can be created via Grafana UI on https://grafana.bioviz.org/ Login->Left panel menu ->explore and write the above query
                Hide
                Permalink
                cdias1 Chester Dias (Inactive) added a comment -

                Attached is screenshot verification of policy for 2 week storage of data in effect

                Show
                cdias1 Chester Dias (Inactive) added a comment - Attached is screenshot verification of policy for 2 week storage of data in effect
                cdias1 Chester Dias (Inactive) made changes -
                Attachment Screenshot 2020-11-13 at 14.28.32.png [ 14940 ]
                Hide
                Permalink
                cdias1 Chester Dias (Inactive) added a comment -

                To Check the policy Run the below query in the explore section of the Grafana UI as per screenshot
                SHOW RETENTION POLICIES ON "Grafana"

                Show
                cdias1 Chester Dias (Inactive) added a comment - To Check the policy Run the below query in the explore section of the Grafana UI as per screenshot SHOW RETENTION POLICIES ON "Grafana"
                cdias1 Chester Dias (Inactive) made changes -
                Status In Progress [ 3 ] Needs 1st Level Review [ 10005 ]
                cdias1 Chester Dias (Inactive) made changes -
                Assignee Chester Dias [ cdias1 ] Nowlan Freese [ nfreese ]
                nfreese Nowlan Freese made changes -
                Status Needs 1st Level Review [ 10005 ] First Level Review in Progress [ 10301 ]
                Hide
                Permalink
                nfreese Nowlan Freese added a comment -

                Chester Dias does the retention policy need to be set as default?

                Show
                nfreese Nowlan Freese added a comment - Chester Dias does the retention policy need to be set as default?
                nfreese Nowlan Freese made changes -
                Status First Level Review in Progress [ 10301 ] Needs 1st Level Review [ 10005 ]
                nfreese Nowlan Freese made changes -
                Assignee Nowlan Freese [ nfreese ] Chester Dias [ cdias1 ]
                nfreese Nowlan Freese made changes -
                Status Needs 1st Level Review [ 10005 ] First Level Review in Progress [ 10301 ]
                nfreese Nowlan Freese made changes -
                Status First Level Review in Progress [ 10301 ] To-Do [ 10305 ]
                Hide
                Permalink
                cdias1 Chester Dias (Inactive) added a comment -

                Nowlan FreeseI am not very sure how the default policy works. I can see no data prior to 12th November in Grafana, I think it is safe to assume the old data is purged. No data of any earlier attempts seems to be present

                Show
                cdias1 Chester Dias (Inactive) added a comment - Nowlan Freese I am not very sure how the default policy works. I can see no data prior to 12th November in Grafana, I think it is safe to assume the old data is purged. No data of any earlier attempts seems to be present
                cdias1 Chester Dias (Inactive) made changes -
                Status To-Do [ 10305 ] In Progress [ 3 ]
                cdias1 Chester Dias (Inactive) made changes -
                Status In Progress [ 3 ] Needs 1st Level Review [ 10005 ]
                cdias1 Chester Dias (Inactive) made changes -
                Assignee Chester Dias [ cdias1 ] Nowlan Freese [ nfreese ]
                nfreese Nowlan Freese made changes -
                Status Needs 1st Level Review [ 10005 ] First Level Review in Progress [ 10301 ]
                Hide
                Permalink
                nfreese Nowlan Freese added a comment -

                Closing issue.

                Show
                nfreese Nowlan Freese added a comment - Closing issue.
                nfreese Nowlan Freese made changes -
                Assignee Nowlan Freese [ nfreese ] Chester Dias [ cdias1 ]
                nfreese Nowlan Freese made changes -
                Status First Level Review in Progress [ 10301 ] Ready for Pull Request [ 10304 ]
                nfreese Nowlan Freese made changes -
                Status Ready for Pull Request [ 10304 ] Pull Request Submitted [ 10101 ]
                nfreese Nowlan Freese made changes -
                Status Pull Request Submitted [ 10101 ] Reviewing Pull Request [ 10303 ]
                nfreese Nowlan Freese made changes -
                Status Reviewing Pull Request [ 10303 ] Merged Needs Testing [ 10002 ]
                nfreese Nowlan Freese made changes -
                Status Merged Needs Testing [ 10002 ] Post-merge Testing In Progress [ 10003 ]
                nfreese Nowlan Freese made changes -
                Resolution Done [ 10000 ]
                Status Post-merge Testing In Progress [ 10003 ] Closed [ 6 ]

                  People

                  • Assignee:
                    cdias1 Chester Dias (Inactive)
                    Reporter:
                    nfreese Nowlan Freese
                  • Votes:
                    0 Vote for this issue
                    Watchers:
                    2 Start watching this issue

                    Dates

                    • Created:
                      Updated:
                      Resolved: