inventory-provider merge requestshttps://gitlab.software.geant.org/geant-swd/dashboardv3/inventory-provider/-/merge_requests2024-03-15T09:55:41Zhttps://gitlab.software.geant.org/geant-swd/dashboardv3/inventory-provider/-/merge_requests/30Feature/pol1 775 poller interfaces cache built from preparsed2024-03-15T09:55:41ZSam RobertsFeature/pol1 775 poller interfaces cache built from preparsedChanges cache generation to use pre-parsed netconf caches in Redis. All unit tests pass as of writing.Changes cache generation to use pre-parsed netconf caches in Redis. All unit tests pass as of writing.Sam RobertsSam Robertshttps://gitlab.software.geant.org/geant-swd/dashboardv3/inventory-provider/-/merge_requests/29Improve testing speed2024-03-13T08:51:48ZPelle KosterImprove testing speedImproves testing speed by ~x3 (from ~3 min to ~1 min on my machine, parallel 12 threads):
* Load big 'router-info.json' only once, instead keep one non-mutating dict for all tests
* mock out ncc client to prevent trying to connect
* do ...Improves testing speed by ~x3 (from ~3 min to ~1 min on my machine, parallel 12 threads):
* Load big 'router-info.json' only once, instead keep one non-mutating dict for all tests
* mock out ncc client to prevent trying to connect
* do not setup logging in tests (mostly for not creating `error.log` and `info.log` files)
* some more minor changes...https://gitlab.software.geant.org/geant-swd/dashboardv3/inventory-provider/-/merge_requests/28[POL1-0703] add support for IC1 dashboard2024-03-12T09:15:41ZPelle Koster[POL1-0703] add support for IC1 dashboardhttps://gitlab.software.geant.org/geant-swd/dashboardv3/inventory-provider/-/merge_requests/27[DBOARD3-676]: Adding circuit types to the list of mic data2024-01-12T13:19:15ZSaket Agrahari[DBOARD3-676]: Adding circuit types to the list of mic datahttps://gitlab.software.geant.org/geant-swd/dashboardv3/inventory-provider/-/merge_requests/26Adding testing for mic2023-12-28T13:37:46ZSaket AgrahariAdding testing for michttps://gitlab.software.geant.org/geant-swd/dashboardv3/inventory-provider/-/merge_requests/25[DBOARD3-676]: Adding changes to add third party id support on inventory prov...2023-12-28T11:08:46ZSaket Agrahari[DBOARD3-676]: Adding changes to add third party id support on inventory provider sidehttps://gitlab.software.geant.org/geant-swd/dashboardv3/inventory-provider/-/merge_requests/24Feature/dboard3 833 handle inactive bundle config2023-11-20T11:01:47ZErik ReidFeature/dboard3 833 handle inactive bundle configDBOARD3-833DBOARD3-833Robert LattaRobert Lattahttps://gitlab.software.geant.org/geant-swd/dashboardv3/inventory-provider/-/merge_requests/23address issues with some speeds not being recognised, a timeout issue not...2023-11-20T11:47:00ZSam Robertsaddress issues with some speeds not being recognised, a timeout issue not...address issues with some speeds not being recognised, a timeout issue not being handled correctly and a bundle formatting erroraddress issues with some speeds not being recognised, a timeout issue not being handled correctly and a bundle formatting errorBjarke MadsenBjarke Madsenhttps://gitlab.software.geant.org/geant-swd/dashboardv3/inventory-provider/-/merge_requests/22Feature/pol1 744 fix speed issues2023-11-06T10:28:56ZSam RobertsFeature/pol1 744 fix speed issuesDoes not include IMS changes (need for this work still unclear), does fix the issues with the speed endpoint.Does not include IMS changes (need for this work still unclear), does fix the issues with the speed endpoint.Sam RobertsSam Robertshttps://gitlab.software.geant.org/geant-swd/dashboardv3/inventory-provider/-/merge_requests/19Feature/pol1 689 router port speeds2023-08-15T13:38:04ZSam RobertsFeature/pol1 689 router port speedsSam RobertsSam Robertshttps://gitlab.software.geant.org/geant-swd/dashboardv3/inventory-provider/-/merge_requests/18Support ASN dot notation.2023-08-07T11:24:01ZNeda MoeiniSupport ASN dot notation.Neda MoeiniNeda Moeinihttps://gitlab.software.geant.org/geant-swd/dashboardv3/inventory-provider/-/merge_requests/16Parallel testing2023-07-28T08:37:21ZBjarke MadsenParallel testingThis merge request allows for tests to be run in parallel, resulting in about a 20x speedup of the test suite.
Locally, tests run to completion in under a minute, with the same coverage as shown in the previous release build process.
...This merge request allows for tests to be run in parallel, resulting in about a 20x speedup of the test suite.
Locally, tests run to completion in under a minute, with the same coverage as shown in the previous release build process.
My local coverage (my worker.py is slightly modified, ignore the difference in statements/coverage for that one):
```
---------- coverage: platform linux, python 3.8.10-final-0 -----------
Name Stmts Miss Cover
--------------------------------------------------------------------
inventory_provider/__init__.py 46 0 100%
inventory_provider/app.py 13 13 0%
inventory_provider/config.py 7 0 100%
inventory_provider/db/__init__.py 0 0 100%
inventory_provider/db/ims.py 188 35 81%
inventory_provider/db/ims_data.py 224 39 83%
inventory_provider/environment.py 25 1 96%
inventory_provider/juniper.py 168 15 91%
inventory_provider/routes/__init__.py 0 0 100%
inventory_provider/routes/classifier.py 482 87 82%
inventory_provider/routes/classifier_schema.py 13 0 100%
inventory_provider/routes/common.py 182 21 88%
inventory_provider/routes/data.py 53 1 98%
inventory_provider/routes/default.py 22 0 100%
inventory_provider/routes/jobs.py 82 6 93%
inventory_provider/routes/lg.py 42 2 95%
inventory_provider/routes/lnetd.py 55 2 96%
inventory_provider/routes/mic.py 16 1 94%
inventory_provider/routes/msr.py 471 58 88%
inventory_provider/routes/neteng.py 45 2 96%
inventory_provider/routes/poller.py 503 14 97%
inventory_provider/routes/testing.py 126 37 71%
inventory_provider/snmp.py 70 9 87%
inventory_provider/tasks/__init__.py 0 0 100%
inventory_provider/tasks/app.py 13 4 69%
inventory_provider/tasks/common.py 101 29 71%
inventory_provider/tasks/config.py 31 5 84%
inventory_provider/tasks/monitor.py 107 13 88%
inventory_provider/tasks/worker.py 792 189 76%
--------------------------------------------------------------------
TOTAL 3877 583 85%
========================================================================= 738 passed, 11 skipped, 11063 warnings in 55.51s =========================================================================
```
The coverage from the last release build:
```
========= 738 passed, 11 skipped, 10978 warnings in 1261.25s (0:21:01) =========
py38: commands[2]> coverage xml
Wrote XML report to coverage.xml
py38: commands[3]> coverage html
Wrote HTML report to htmlcov/index.html
py38: commands[4]> coverage report --fail-under 80
Name Stmts Miss Cover
--------------------------------------------------------------------
inventory_provider/__init__.py 46 0 100%
inventory_provider/app.py 13 13 0%
inventory_provider/config.py 7 0 100%
inventory_provider/db/__init__.py 0 0 100%
inventory_provider/db/ims.py 188 35 81%
inventory_provider/db/ims_data.py 224 39 83%
inventory_provider/environment.py 25 1 96%
inventory_provider/juniper.py 168 15 91%
inventory_provider/routes/__init__.py 0 0 100%
inventory_provider/routes/classifier.py 482 87 82%
inventory_provider/routes/classifier_schema.py 13 0 100%
inventory_provider/routes/common.py 182 21 88%
inventory_provider/routes/data.py 53 1 98%
inventory_provider/routes/default.py 22 0 100%
inventory_provider/routes/jobs.py 82 6 93%
inventory_provider/routes/lg.py 42 2 95%
inventory_provider/routes/lnetd.py 55 2 96%
inventory_provider/routes/mic.py 16 1 94%
inventory_provider/routes/msr.py 471 58 88%
inventory_provider/routes/neteng.py 45 2 96%
inventory_provider/routes/poller.py 503 14 97%
inventory_provider/routes/testing.py 126 37 71%
inventory_provider/snmp.py 70 9 87%
inventory_provider/tasks/__init__.py 0 0 100%
inventory_provider/tasks/app.py 13 4 69%
inventory_provider/tasks/common.py 101 29 71%
inventory_provider/tasks/config.py 31 5 84%
inventory_provider/tasks/monitor.py 107 13 88%
inventory_provider/tasks/worker.py 786 183 77%
--------------------------------------------------------------------
TOTAL 3871 577 85%
```Erik ReidErik Reidhttps://gitlab.software.geant.org/geant-swd/dashboardv3/inventory-provider/-/merge_requests/15feature/DBOARD3-771/include all sites in inventory provider /neteng/pops and ...2023-07-14T10:06:27ZMohammad Torkashvandfeature/DBOARD3-771/include all sites in inventory provider /neteng/pops and /neteng/pop/<lookup>Mohammad TorkashvandMohammad Torkashvandhttps://gitlab.software.geant.org/geant-swd/dashboardv3/inventory-provider/-/merge_requests/14only build snmp indices once per endpoint request2023-06-14T11:15:42ZSam Robertsonly build snmp indices once per endpoint requestSam RobertsSam Robertshttps://gitlab.software.geant.org/geant-swd/dashboardv3/inventory-provider/-/merge_requests/12Add redis auth credentials to celery backend config.2023-06-08T11:34:19ZNeda MoeiniAdd redis auth credentials to celery backend config.Neda MoeiniNeda Moeinihttps://gitlab.software.geant.org/geant-swd/dashboardv3/inventory-provider/-/merge_requests/11Added redis auth credentials to inventory config schema and updated _get_redi...2023-06-07T15:01:02ZNeda MoeiniAdded redis auth credentials to inventory config schema and updated _get_redis wrapper.Neda MoeiniNeda Moeinihttps://gitlab.software.geant.org/geant-swd/dashboardv3/inventory-provider/-/merge_requests/9Add sentry SDK to flask app and celery worker2023-05-30T14:29:33ZBjarke MadsenAdd sentry SDK to flask app and celery workerRobert LattaRobert Lattahttps://gitlab.software.geant.org/geant-swd/dashboardv3/inventory-provider/-/merge_requests/8parse et interfaces with speed of 400 GB2023-04-25T10:13:06ZSam Robertsparse et interfaces with speed of 400 GBSam RobertsSam Robertshttps://gitlab.software.geant.org/geant-swd/dashboardv3/inventory-provider/-/merge_requests/7Add fix for assigning correct port SIDs to circuits2023-04-13T14:33:02ZBjarke MadsenAdd fix for assigning correct port SIDs to circuitsErik ReidErik Reidhttps://gitlab.software.geant.org/geant-swd/dashboardv3/inventory-provider/-/merge_requests/6Expect whitespace in interface description prefix matching.2022-11-24T12:48:29ZBjarke MadsenExpect whitespace in interface description prefix matching.Ensures matches are not partial in case of overlapping naming schemesEnsures matches are not partial in case of overlapping naming schemesErik ReidErik Reid