inventory-provider merge requestshttps://gitlab.software.geant.org/geant-swd/dashboardv3/inventory-provider/-/merge_requests2022-07-14T10:56:37Zhttps://gitlab.software.geant.org/geant-swd/dashboardv3/inventory-provider/-/merge_requests/1Feature/reporting 297 msr mdvpn endpoint2022-07-14T10:56:37ZSam RobertsFeature/reporting 297 msr mdvpn endpointSam RobertsSam Robertshttps://gitlab.software.geant.org/geant-swd/dashboardv3/inventory-provider/-/merge_requests/2Feature/reporting 307 msr vpn proxy endpoint2022-07-14T12:39:33ZSam RobertsFeature/reporting 307 msr vpn proxy endpointAdds /msr/vpn-proxy endpointAdds /msr/vpn-proxy endpointSam RobertsSam Robertshttps://gitlab.software.geant.org/geant-swd/dashboardv3/inventory-provider/-/merge_requests/3Feature/reporting 311 add msr asn peers2022-08-03T15:12:06ZSam RobertsFeature/reporting 311 add msr asn peersReady for merge (pending additional functionality wrt caching and NRENs)Ready for merge (pending additional functionality wrt caching and NRENs)Sam RobertsSam Robertshttps://gitlab.software.geant.org/geant-swd/dashboardv3/inventory-provider/-/merge_requests/4Add feature to raise SIDs defined on ports to services on those ports2022-08-19T13:40:58ZBjarke MadsenAdd feature to raise SIDs defined on ports to services on those portsAn SID defined on a port will only be assigned to a service if a single service exists on that port.An SID defined on a port will only be assigned to a service if a single service exists on that port.Erik ReidErik Reidhttps://gitlab.software.geant.org/geant-swd/dashboardv3/inventory-provider/-/merge_requests/5Add port_type parsing from description for BRIAN2022-11-18T11:32:57ZBjarke MadsenAdd port_type parsing from description for BRIANErik ReidErik Reidhttps://gitlab.software.geant.org/geant-swd/dashboardv3/inventory-provider/-/merge_requests/6Expect whitespace in interface description prefix matching.2022-11-24T12:48:29ZBjarke MadsenExpect whitespace in interface description prefix matching.Ensures matches are not partial in case of overlapping naming schemesEnsures matches are not partial in case of overlapping naming schemesErik ReidErik Reidhttps://gitlab.software.geant.org/geant-swd/dashboardv3/inventory-provider/-/merge_requests/7Add fix for assigning correct port SIDs to circuits2023-04-13T14:33:02ZBjarke MadsenAdd fix for assigning correct port SIDs to circuitsErik ReidErik Reidhttps://gitlab.software.geant.org/geant-swd/dashboardv3/inventory-provider/-/merge_requests/8parse et interfaces with speed of 400 GB2023-04-25T10:13:06ZSam Robertsparse et interfaces with speed of 400 GBSam RobertsSam Robertshttps://gitlab.software.geant.org/geant-swd/dashboardv3/inventory-provider/-/merge_requests/9Add sentry SDK to flask app and celery worker2023-05-30T14:29:33ZBjarke MadsenAdd sentry SDK to flask app and celery workerRobert LattaRobert Lattahttps://gitlab.software.geant.org/geant-swd/dashboardv3/inventory-provider/-/merge_requests/10Draft: Add redis auth support to inventory dashboard2023-06-07T12:16:10ZNeda MoeiniDraft: Add redis auth support to inventory dashboardNeda MoeiniNeda Moeinihttps://gitlab.software.geant.org/geant-swd/dashboardv3/inventory-provider/-/merge_requests/11Added redis auth credentials to inventory config schema and updated _get_redi...2023-06-07T15:01:02ZNeda MoeiniAdded redis auth credentials to inventory config schema and updated _get_redis wrapper.Neda MoeiniNeda Moeinihttps://gitlab.software.geant.org/geant-swd/dashboardv3/inventory-provider/-/merge_requests/12Add redis auth credentials to celery backend config.2023-06-08T11:34:19ZNeda MoeiniAdd redis auth credentials to celery backend config.Neda MoeiniNeda Moeinihttps://gitlab.software.geant.org/geant-swd/dashboardv3/inventory-provider/-/merge_requests/13Feature/dboard 743 build cache in worker process2023-06-14T10:42:26ZSam RobertsFeature/dboard 743 build cache in worker processSam RobertsSam Robertshttps://gitlab.software.geant.org/geant-swd/dashboardv3/inventory-provider/-/merge_requests/14only build snmp indices once per endpoint request2023-06-14T11:15:42ZSam Robertsonly build snmp indices once per endpoint requestSam RobertsSam Robertshttps://gitlab.software.geant.org/geant-swd/dashboardv3/inventory-provider/-/merge_requests/15feature/DBOARD3-771/include all sites in inventory provider /neteng/pops and ...2023-07-14T10:06:27ZMohammad Torkashvandfeature/DBOARD3-771/include all sites in inventory provider /neteng/pops and /neteng/pop/<lookup>Mohammad TorkashvandMohammad Torkashvandhttps://gitlab.software.geant.org/geant-swd/dashboardv3/inventory-provider/-/merge_requests/16Parallel testing2023-07-28T08:37:21ZBjarke MadsenParallel testingThis merge request allows for tests to be run in parallel, resulting in about a 20x speedup of the test suite.
Locally, tests run to completion in under a minute, with the same coverage as shown in the previous release build process.
...This merge request allows for tests to be run in parallel, resulting in about a 20x speedup of the test suite.
Locally, tests run to completion in under a minute, with the same coverage as shown in the previous release build process.
My local coverage (my worker.py is slightly modified, ignore the difference in statements/coverage for that one):
```
---------- coverage: platform linux, python 3.8.10-final-0 -----------
Name Stmts Miss Cover
--------------------------------------------------------------------
inventory_provider/__init__.py 46 0 100%
inventory_provider/app.py 13 13 0%
inventory_provider/config.py 7 0 100%
inventory_provider/db/__init__.py 0 0 100%
inventory_provider/db/ims.py 188 35 81%
inventory_provider/db/ims_data.py 224 39 83%
inventory_provider/environment.py 25 1 96%
inventory_provider/juniper.py 168 15 91%
inventory_provider/routes/__init__.py 0 0 100%
inventory_provider/routes/classifier.py 482 87 82%
inventory_provider/routes/classifier_schema.py 13 0 100%
inventory_provider/routes/common.py 182 21 88%
inventory_provider/routes/data.py 53 1 98%
inventory_provider/routes/default.py 22 0 100%
inventory_provider/routes/jobs.py 82 6 93%
inventory_provider/routes/lg.py 42 2 95%
inventory_provider/routes/lnetd.py 55 2 96%
inventory_provider/routes/mic.py 16 1 94%
inventory_provider/routes/msr.py 471 58 88%
inventory_provider/routes/neteng.py 45 2 96%
inventory_provider/routes/poller.py 503 14 97%
inventory_provider/routes/testing.py 126 37 71%
inventory_provider/snmp.py 70 9 87%
inventory_provider/tasks/__init__.py 0 0 100%
inventory_provider/tasks/app.py 13 4 69%
inventory_provider/tasks/common.py 101 29 71%
inventory_provider/tasks/config.py 31 5 84%
inventory_provider/tasks/monitor.py 107 13 88%
inventory_provider/tasks/worker.py 792 189 76%
--------------------------------------------------------------------
TOTAL 3877 583 85%
========================================================================= 738 passed, 11 skipped, 11063 warnings in 55.51s =========================================================================
```
The coverage from the last release build:
```
========= 738 passed, 11 skipped, 10978 warnings in 1261.25s (0:21:01) =========
py38: commands[2]> coverage xml
Wrote XML report to coverage.xml
py38: commands[3]> coverage html
Wrote HTML report to htmlcov/index.html
py38: commands[4]> coverage report --fail-under 80
Name Stmts Miss Cover
--------------------------------------------------------------------
inventory_provider/__init__.py 46 0 100%
inventory_provider/app.py 13 13 0%
inventory_provider/config.py 7 0 100%
inventory_provider/db/__init__.py 0 0 100%
inventory_provider/db/ims.py 188 35 81%
inventory_provider/db/ims_data.py 224 39 83%
inventory_provider/environment.py 25 1 96%
inventory_provider/juniper.py 168 15 91%
inventory_provider/routes/__init__.py 0 0 100%
inventory_provider/routes/classifier.py 482 87 82%
inventory_provider/routes/classifier_schema.py 13 0 100%
inventory_provider/routes/common.py 182 21 88%
inventory_provider/routes/data.py 53 1 98%
inventory_provider/routes/default.py 22 0 100%
inventory_provider/routes/jobs.py 82 6 93%
inventory_provider/routes/lg.py 42 2 95%
inventory_provider/routes/lnetd.py 55 2 96%
inventory_provider/routes/mic.py 16 1 94%
inventory_provider/routes/msr.py 471 58 88%
inventory_provider/routes/neteng.py 45 2 96%
inventory_provider/routes/poller.py 503 14 97%
inventory_provider/routes/testing.py 126 37 71%
inventory_provider/snmp.py 70 9 87%
inventory_provider/tasks/__init__.py 0 0 100%
inventory_provider/tasks/app.py 13 4 69%
inventory_provider/tasks/common.py 101 29 71%
inventory_provider/tasks/config.py 31 5 84%
inventory_provider/tasks/monitor.py 107 13 88%
inventory_provider/tasks/worker.py 786 183 77%
--------------------------------------------------------------------
TOTAL 3871 577 85%
```Erik ReidErik Reidhttps://gitlab.software.geant.org/geant-swd/dashboardv3/inventory-provider/-/merge_requests/17Draft: implement new port speeds2023-08-01T13:29:41ZBjarke MadsenDraft: implement new port speedshttps://gitlab.software.geant.org/geant-swd/dashboardv3/inventory-provider/-/merge_requests/18Support ASN dot notation.2023-08-07T11:24:01ZNeda MoeiniSupport ASN dot notation.Neda MoeiniNeda Moeinihttps://gitlab.software.geant.org/geant-swd/dashboardv3/inventory-provider/-/merge_requests/19Feature/pol1 689 router port speeds2023-08-15T13:38:04ZSam RobertsFeature/pol1 689 router port speedsSam RobertsSam Robertshttps://gitlab.software.geant.org/geant-swd/dashboardv3/inventory-provider/-/merge_requests/20Don't validate joblog:cached-response values2023-09-04T12:31:38ZBjarke MadsenDon't validate joblog:cached-response valuesWe constantly get jsonschema.ValidationError on the joblog:cached-response key, because it doesn't follow the structure of other joblogs:
```json
{"errors":[],"failed":[],"pending":[],"warnings":["'Returning cached interface info data f...We constantly get jsonschema.ValidationError on the joblog:cached-response key, because it doesn't follow the structure of other joblogs:
```json
{"errors":[],"failed":[],"pending":[],"warnings":["'Returning cached interface info data for mx1.gen.ch.geant.net'","'error loading interface-info data from mx1.gen.ch.geant.net'","'no cached interface info for intinfo:srx2.am.office.geant.net'","'error loading interface-info data from srx2.am.office.geant.net'","'error loading interface-info data from srx2.ch.office.geant.net'","'no cached interface info for intinfo:srx2.ch.office.geant.net'","'Returning cached interface info data for qfx.par.fr.geant.net'","'error loading interface-info data from qfx.par.fr.geant.net'","'error loading interface-info data from srx1.ch.office.geant.net'","'no cached interface info for intinfo:srx1.ch.office.geant.net'"]}
```Erik ReidErik Reid