Commit a6650dd9 authored by Marco Malavolti's avatar Marco Malavolti
Browse files

ECCS2 First Working release

parent 2c27af66
......@@ -8,12 +8,12 @@
6. [Requirements Hardware](#requirements-hardware)
7. [Requirements Software](#requirements-software)
8. [HOWTO Install and Configure](#howto-install-and-configure)
* [Install Python 3.8.x](#install-python-38x)
* [Install Python 3.9.x](#install-python-39x)
+ [CentOS 7 requirements](#centos-7-requirements)
+ [Debian requirements](#debian-requirements)
+ [Python 3.8](#python-38)
+ [Python 3.9](#python-39)
9. [Install the Chromedriver](#install-the-chromedriver)
10. [Install Chromium needed by Selenium](#install-chromium-needed-by-selenium)
10. [Install Google Chrome needed by Selenium](#install-google-chrome-needed-by-selenium)
11. [ECCS2 Script](#eccs2-script)
* [Install](#install)
* [Configure](#configure)
......@@ -84,12 +84,15 @@ The tool uses following status for IdPs:
* HDD: 10 GB
* RAM: 4 GB
* CPU: >= 2 vCPU (suggested)
* ARCH: 64 Bit
# Requirements Software
* Apache Server + WSGI
* Python 3.8 (tested with v3.8.3,v3.8.5)
* Selenim + Chromium Web Brower
* Python 3.9 (tested with v3.9.6)
* Selenim + Google Chrome Web Brower (tested with v91.0.4472.164)
* Chromedriver (tested with v91.0.4472.101)
* Git
# HOWTO Install and Configure
......@@ -97,7 +100,7 @@ The tool uses following status for IdPs:
* `cd $HOME ; git clone https://github.com/malavolti/eccs2.git`
## Install Python 3.8.x
## Install Python 3.9.x
### CentOS 7 requirements
......@@ -110,6 +113,9 @@ The tool uses following status for IdPs:
3. Install needed packages to build python:
* `sudo yum -y install openssl-devel bzip2-devel libffi-devel wget`
4. Install Git:
* `sudo yum -y install git`
### Debian requirements
1. Update the system packages:
......@@ -118,45 +124,49 @@ The tool uses following status for IdPs:
2. Install needed packages to build python:
* `sudo apt install build-essential zlib1g-dev libncurses5-dev libgdbm-dev libnss3-dev libssl-dev libreadline-dev libffi-dev`
### Python 3.8
3. Install Git:
* `sudo apt install git`
### Python 3.9
1. Download the last version of Python 3.8.x from https://www.python.org/downloads/source/ into your home:
* `wget https://www.python.org/ftp/python/3.8.5/Python-3.8.5.tgz -O $HOME/eccs2/Python-3.8.5.tgz`
1. Download the last version of Python 3.9.x from https://www.python.org/downloads/source/ into your home:
* `wget https://www.python.org/ftp/python/3.9.6/Python-3.9.6.tgz -O $HOME/eccs2/Python-3.9.6.tgz`
2. Extract Python source package:
* `cd $HOME/eccs2/`
* `tar xzf Python-3.8.5.tgz`
* `tar xzf Python-3.9.6.tgz`
3. Build Python from the source package:
* `cd $HOME/eccs2/Python-3.8.5`
* `cd $HOME/eccs2/Python-3.9.6`
* `./configure --prefix=$HOME/eccs2/python`
* `make`
4. Install Python 3.8.x under `$HOME/eccs2/python`:
4. Install Python 3.9.x under `$HOME/eccs2/python`:
* `make install`
* `$HOME/eccs2/python/bin/python3.8 --version`
* `$HOME/eccs2/python/bin/python3.9 --version`
This will install python under your $HOME directory.
5. Remove useless things:
* `rm -Rf $HOME/eccs2/Python-3.8.5 $HOME/eccs2/Python-3.8.5.tgz`
* `rm -Rf $HOME/eccs2/Python-3.9.9 $HOME/eccs2/Python-3.9.6.tgz`
# Install Chromium needed by Selenium
# Install Google Chrome needed by Selenium
* Debian:
* `sudo apt install chromium git jq`
* Debian (64 bit):
* `sudo wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb`
* `sudo apt install ./google-chrome-stable_current_amd64.deb`
* CentOS:
* `sudo yum install -y epel-release`
* `sudo yum install -y chromium git jq`
* CentOS (64 bit):
* `sudo wget https://dl.google.com/linux/direct/google-chrome-stable_current_x86_64.rpm`
* `sudo yum install ./google-chrome-stable_current_x86_64.rpm`
# Install the Chromedriver
1. Find out which version of Chromium you are using:
* Debian 9 (stretch):
* `chromium -version` => Chromium 73.0.3683.75
* `google-chrome -version` => Google Chrome 91.0.4472.164
* CentOS 7.8:
* `chromium-browser -version` => Chromium 83.0.4103.116
* `google-chrome -version` => Google Chrome 91.0.4472.164
2. Take the Chrome version number, remove the last part, and append the result to URL "`https://chromedriver.storage.googleapis.com/LATEST_RELEASE_`". For example, with Chrome version 73.0.3683.75, you'd get a URL "`https://chromedriver.storage.googleapis.com/LATEST_RELEASE_73.0.3683`".
......@@ -178,8 +188,8 @@ After the initial download, it is recommended that you occasionally go through t
## Install
* `cd $HOME/eccs2`
* `./python/bin/python3.8 -m pip install virtualenv`
* `$HOME/eccs2/python/bin/virtualenv --python=$HOME/eccs2/python/bin/python3.8 eccs2venv`
* `./python/bin/python3.9 -m pip install virtualenv`
* `$HOME/eccs2/python/bin/virtualenv --python=$HOME/eccs2/python/bin/python3.9 eccs2venv`
* `source eccs2venv/bin/activate` (`deactivate` to exit Virtualenv)
* `python -m pip install -r requirements.txt`
......@@ -228,11 +238,11 @@ After the initial download, it is recommended that you occasionally go through t
* `./runEccs2.py --idp <IDP-ENTITYID>` (to run check on a single IdP)
* `./runEccs2.py --test` (to run a full check without effects)
* `./runEccs2.py --idp <IDP-ENTITYID> --test` (to run check on a single IdP without effects)
* `./runEccs2.py --idp <IDP-ENTITYID> --replace` (to run check on a single IdP and replace, or add, a result)
The check will run a second time for those IdPs that failed the first execution of the script.
If something prevent the good execution of the ECCS2's check, the `logs/failed-cmd.sh` file will be not empty at the end of the execution.
The "--test" parameter will not change the result of ECCS2, but will write the output on the `logs/stdout_idp_YYYY-MM-DD.log`,`logs/stderr_idp_YYYY-MM-DD.log` and `logs/failed-cmd-idp.sh` files.
The "--test" parameter will not change the result of ECCS2, but will write the output on the `logs/stdout_idp_YYYY-MM-DD.log`,`logs/stderr_idp_YYYY-MM-DD.log` and `logs/failed-cmd-idp.sh` files if the argument "--test" will be used.
# ECCS2 API Server (uWSGI)
......
......@@ -7,7 +7,7 @@ import re
from eccs2properties import DAY, ECCS2LOGSDIR, ECCS2OUTPUTDIR, ECCS2LISTFEDSURL, ECCS2LISTFEDSFILE
from flask import Flask, request, jsonify
from flask_restful import Resource, Api
from utils import getLogger, getListFeds, getRegAuthDict
from utils import get_logger, get_list_feds, get_reg_auth_dict
app = Flask(__name__)
api = Api(app)
......@@ -175,8 +175,8 @@ class EccsResults(Resource):
# /api/fedstats
class FedStats(Resource):
def get(self):
list_feds = getListFeds(ECCS2LISTFEDSURL, ECCS2LISTFEDSFILE)
regAuthDict = getRegAuthDict(list_feds)
list_feds = get_list_feds(ECCS2LISTFEDSURL, ECCS2LISTFEDSFILE)
regAuthDict = get_reg_auth_dict(list_feds)
file_path = "%s/eccs2_%s.log" % (ECCS2OUTPUTDIR,DAY)
date = DAY
......@@ -263,5 +263,5 @@ if __name__ == '__main__':
# Useful only for API development Server
#app.config['JSON_AS_ASCII'] = True
#app.logger.removeHandler(default_handler)
#app.logger = getLogger("eccs2api.log", ECCS2LOGSDIR, "w", "INFO")
#app.logger = get_logger("eccs2api.log", ECCS2LOGSDIR, "w", "INFO")
app.run(port='5002')
......@@ -11,61 +11,3 @@ rm -f $BASEDIR/eccs2/input/*.json
# Run ECCS2
$BASEDIR/eccs2/runEccs2.py
# Run Failed Command again
bash $BASEDIR/eccs2/logs/failed-cmd.sh
date=$(date '+%Y-%m-%d')
file="$BASEDIR/eccs2/logs/failed-cmd.sh"
prefix="$BASEDIR/eccs2/eccs2.py '"
suffix="'"
eccs2output="$BASEDIR/eccs2/output/eccs2_$date.log"
declare -a eccs2cmdToRemoveArray
# If the ECCS2 output contains the result of a failed command (failed-cmd.sh),
# than remove the failed command from the failed-cmd.sh file
if [ -s $eccsoutput ]; then
if [ -s $file ]; then
while IFS= read -r line
do
string=$line
#remove "prefix" from the command string at the beginning.
prefix_removed_string=${string/#$prefix}
#remove "suffix" from the command string at the end.
suffix_removed_string=${prefix_removed_string/%$suffix}
entityIDidp=$(echo "$suffix_removed_string" | jq '.entityID')
#remove start and end quotes from the entityIDidp to be able to use "grep"
entityIDidp="${entityIDidp:1}"
entityIDidp="${entityIDidp%?}"
result=$(grep $entityIDidp $eccs2output | wc -l)
if [[ "$result" = 1 ]]; then
eccs2cmdToRemoveArray+=("$entityIDidp")
else
echo "The result for the IdP '$entityIDidp' has been found multiple times on $eccs2output. It is wrong."
fi
done <"$file"
# Remove IdP command that had success from "failed-cmd.sh"
for idpToRemove in ${eccs2cmdToRemoveArray[@]}
do
$(grep -v $idpToRemove $file > temp ; mv -f temp $file)
done
if [ -s $file ]; then
echo "$date - ECCS2 NOT OK: Some eduGAIN IdPs have remained unchecked. See the 'logs/failed-cmd.sh' and logs/stderr_$date.log files"
else
echo "$date - ECCS2 OK: All eduGAIN IdPs have been checked successfully"
fi
else
echo "$date - ECCS2 OK: All eduGAIN IdPs have been checked successfully"
fi
else
echo "$date - Something went wrong and the ECCS2 check has not been executed"
fi
#!/usr/bin/env python3
import argparse
import datetime
import json
import re
import requests
import sys
import utils
import eccs2properties as e2p
from eccs2properties import DAY, ECCS2HTMLDIR, ECCS2OUTPUTDIR, ECCS2RESULTSLOG, ECCS2SPS, ECCS2SELENIUMDEBUG,ROBOTS_USER_AGENT,ECCS2REQUESTSTIMEOUT, FEDS_DISABLED_DICT, IDPS_DISABLED_DICT, ECCS2SELENIUMPAGELOADTIMEOUT
from pathlib import Path
from selenium.common.exceptions import TimeoutException
from urllib3.util import parse_url
from utils import getLogger, getIdPContacts, getDriver
"""
The check works with the wayfless url of two SP and successed if the IdP Login Page appears and contains the fields "username" and "password" for each of them.
It is possible to disable the check by eccs2properties with *denylist or by "robots.txt" put on the SAMLRequest endpoint root web dir.
It is possible to disable the check by eccs2properties IDP_DISABLE_DICT or by "robots.txt" put on the SAMLRequest endpoint root web dir.
"""
# Returns the FQDN to use on the HTML page_source files
def getIDPlabel(url_or_urn):
if url_or_urn.startswith('http'):
return parse_url(url_or_urn)[2]
else:
return url_or_urn.split(":")[-1]
def getIDPfqdn(samlrequest_url):
return getIDPlabel(samlrequest_url)
# This function checks if an IdP recognized the SP by presenting its Login page with "username" and "password" fields.
# It is possible to disable the check on eccs2properties with the *denylist or by "robots.txt" file into the SAMLRequest endpoint root web dir.
# If the IdP Login page contains "username" and "password" fields the test is passed.
def checkIdP(sp,idp,test):
# Disable SSL requests warning messages
requests.packages.urllib3.disable_warnings()
debug_selenium = ECCS2SELENIUMDEBUG
label_idp = getIDPlabel(idp['entityID'])
# WebDriver MUST be instanced here to avoid problems with SESSION
driver = getDriver(label_idp,debug_selenium)
# Exception of WebDriver raises
if (driver == None):
return None
# Configure Blacklists
federations_disabled_dict = FEDS_DISABLED_DICT
idps_disabled_dict = IDPS_DISABLED_DICT
fqdn_sp = parse_url(sp)[2]
wayfless_url = sp + idp['entityID']
robots = ""
if (idp['registrationAuthority'] in federations_disabled_dict.keys()):
check_time = datetime.datetime.utcnow().strftime('%Y-%m-%dT%H:%M:%S') + 'Z'
if (test is not True):
with open("%s/%s/%s---%s.html" % (ECCS2HTMLDIR,DAY,label_idp,fqdn_sp),"w") as html:
html.write("%s" % federations_disabled_dict[idp['registrationAuthority']])
else:
print("%s" % federations_disabled_dict[idp['registrationAuthority']])
return (idp['entityID'],wayfless_url,check_time,"NULL","DISABLED")
if (idp['entityID'] in idps_disabled_dict.keys()):
check_time = datetime.datetime.utcnow().strftime('%Y-%m-%dT%H:%M:%S') + 'Z'
if (test is not True):
with open("%s/%s/%s---%s.html" % (ECCS2HTMLDIR,DAY,label_idp,fqdn_sp),"w") as html:
html.write("%s" % idps_disabled_dict[idp['entityID']])
else:
print("%s" % idps_disabled_dict[idp['entityID']])
return (idp['entityID'],wayfless_url,check_time,"NULL","DISABLED")
# Open SP via wayfless_url and reach the IdP login page to check
try:
check_time = datetime.datetime.utcnow().strftime('%Y-%m-%dT%H:%M:%S') + 'Z'
driver.get(wayfless_url)
page_source = driver.page_source
samlrequest_url = driver.current_url
if (test is not True):
# Put the page_source into an appropriate HTML file
with open("%s/%s/%s---%s.html" % (ECCS2HTMLDIR,DAY,label_idp,fqdn_sp),"w") as html:
html.write(page_source)
else:
print("\n[page_source of '%s' for sp '%s']\n%s" % (label_idp,fqdn_sp,page_source))
except TimeoutException as e:
if (test is not True):
# Put an empty string into the page_source file
with open("%s/%s/%s---%s.html" % (ECCS2HTMLDIR,DAY,label_idp,fqdn_sp),"w") as html:
html.write("<html><h1>The IdP Login page was not loaded within %d seconds.</h1></html>" % ECCS2SELENIUMPAGELOADTIMEOUT )
else:
print("\n[page_source of '%s' for sp '%s']\nNo source code" % (label_idp,fqdn_sp))
return (idp['entityID'],wayfless_url,check_time,"(failed)","Timeout")
except Exception as e:
print ("!!! EXCEPTION DRIVER !!!")
print (e.__str__())
print ("IdP: %s\nSP: %s" % (idp['entityID'],sp))
return None
finally:
driver.quit()
try:
headers = {
'User-Agent': '%s' % ROBOTS_USER_AGENT
}
fqdn_idp = getIDPfqdn(samlrequest_url)
robots = requests.get("https://%s/robots.txt" % fqdn_idp, headers=headers, verify=True, timeout=ECCS2REQUESTSTIMEOUT)
if (robots == ""):
robots = requests.get("http://%s/robots.txt" % fqdn_idp, headers=headers, verify=False, timeout=ECCS2REQUESTSTIMEOUT)
# Catch SSL Exceptions and block the ECCS check
except (requests.exceptions.SSLError) as e:
check_time = datetime.datetime.utcnow().strftime('%Y-%m-%dT%H:%M:%S') + 'Z'
if (test is not True):
with open("%s/%s/%s---%s.html" % (ECCS2HTMLDIR,DAY,label_idp,fqdn_sp),"w") as html:
html.write("<p>IdP excluded from check due the following SSL Error:<br/><br/>%s</p><p>Check it on SSL Labs: <a href='https://www.ssllabs.com/ssltest/analyze.html?d=%s'>Click Here</a></p>" % (e.__str__(),fqdn_idp))
else:
print("IdP excluded from check due the following SSL Error:\n\n%s\n\nCheck it on SSL Labs: https://www.ssllabs.com/ssltest/analyze.html?d=%s" % (e.__str__(),fqdn_idp))
return (idp['entityID'],wayfless_url,check_time,"(failed)","SSL-Error")
# Pass every other exceptions on /robots.txt file. Consider only SSL Exceptions.
except Exception as e:
#print("IdP '%s' HAD HAD A REQUEST ERROR: %s" % (fqdn_idp,e.__str__()))
robots = ""
if (robots):
check_time = datetime.datetime.utcnow().strftime('%Y-%m-%dT%H:%M:%S') + 'Z'
p = re.compile('^User-agent:\sECCS\sDisallow:\s\/\s*$', re.MULTILINE)
m = p.search(robots.text)
if (m):
if (test is not True):
with open("%s/%s/%s---%s.html" % (ECCS2HTMLDIR,DAY,label_idp,fqdn_sp),"w") as html:
html.write("IdP excluded from check by robots.txt")
else:
print("IdP excluded from check by robots.txt")
return (idp['entityID'],wayfless_url,check_time,"NULL","DISABLED")
pattern_metadata = "Unable.to.locate(\sissuer.in|).metadata(\sfor|)|no.metadata.found|profile.is.not.configured.for.relying.party|Cannot.locate.entity|fail.to.load.unknown.provider|does.not.recognise.the.service|unable.to.load.provider|Nous.n'avons.pas.pu.(charg|charger).le.fournisseur.de service|Metadata.not.found|application.you.have.accessed.is.not.registered.for.use.with.this.service|Message.did.not.meet.security.requirements"
pattern_username = '<input[\s]+[^>]*((type=\s*[\'"](text|email)[\'"]|user)|(name=\s*[\'"](name)[\'"]))[^>]*>';
pattern_password = '<input[\s]+[^>]*(type=\s*[\'"]password[\'"]|password)[^>]*>';
metadata_not_found = re.search(pattern_metadata,page_source, re.I)
username_found = re.search(pattern_username,page_source, re.I)
password_found = re.search(pattern_password,page_source, re.I)
try:
headers = {'User-Agent':'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.61 Safari/537.36'}
http_code = str(requests.get(samlrequest_url, headers=headers, verify=False, timeout=ECCS2REQUESTSTIMEOUT).status_code)
except requests.exceptions.ConnectionError as e:
print ("http-code: (failed) - ConnectionError for IdP '%s' with SP '%s'" % (idp['entityID'],sp))
#print("!!! REQUESTS HTTP CODE CONNECTION ERROR EXCEPTION !!!")
#print (e.__str__())
http_code = "(failed)"
except requests.exceptions.Timeout as e:
print ("http-code: (failed) - TimeoutError for IdP '%s' with SP '%s'" % (idp['entityID'],sp))
#print("!!! REQUESTS HTTP CODE TIMEOUT EXCEPTION !!!")
#print (e.__str__())
http_code = "(failed)"
except requests.exceptions.TooManyRedirects as e:
print ("http-code: (failed) - TooManyRedirectsError for IdP '%s' with SP '%s'" % (idp['entityID'],sp))
#print("!!! REQUESTS HTTP CODE TOO MANY REDIRECTS EXCEPTION !!!")
#print (e.__str__())
http_code = "(failed)"
except requests.exceptions.RequestException as e:
print ("http-code: (failed) - RequestException for IdP '%s' with SP '%s'" % (idp['entityID'],sp))
#print ("!!! REQUESTS EXCEPTION !!!")
print (e.__str__())
http_code = "(failed)"
except Exception as e:
print ("http-code: (failed) - OtherException for IdP '%s' with SP '%s'" % (idp['entityID'],sp))
#print ("!!! EXCEPTION REQUESTS !!!")
print (e.__str__())
http_code = "(failed)"
if(metadata_not_found):
return (idp['entityID'],wayfless_url,check_time,http_code,"No-eduGAIN-Metadata")
elif not username_found or not password_found:
return (idp['entityID'],wayfless_url,check_time,http_code,"Invalid-Form")
else:
return (idp['entityID'],wayfless_url,check_time,http_code,"OK")
# Extract IdP DisplayName by fixing input string
def getDisplayName(display_name):
def get_display_name(display_name):
display_name_equal_splitted = display_name.split('==')
for elem in display_name_equal_splitted:
if "en" in elem:
......@@ -216,39 +24,40 @@ def getDisplayName(display_name):
elem = elem.replace('"','\\"')
return elem.split(';', 1)[1]
# Append the result of the check on a file
def storeECCS2result(idp,check_results,idp_status,test):
def store_eccs_result(idp,sp,check_results,idp_status,test):
# Build the contacts lists: technical/support
list_technical_contacts = getIdPContacts(idp,'technical')
list_support_contacts = getIdPContacts(idp,'support')
list_technical_contacts = utils.get_idp_contacts(idp,'technical')
list_support_contacts = utils.get_idp_contacts(idp,'support')
str_technical_contacts = ','.join(list_technical_contacts)
str_support_contacts = ','.join(list_support_contacts)
if (test is not True):
# IdP-DisplayName;IdP-entityID;IdP-RegAuth;IdP-tech-ctc-1,IdP-tech-ctc-2;IdP-supp-ctc-1,IdP-supp-ctc-2;IdP-ECCS-Status;SP-wayfless-url-1;SP-check-time-1;SP-http-code-1;SP-result-1;SP-wayfless-url-2;SP-check-time-2;SP-http-code-2;SP-result-2
with open("%s/%s" % (ECCS2OUTPUTDIR,ECCS2RESULTSLOG), 'a') as f:
f.write('{"displayName":"%s","entityID":"%s","registrationAuthority":"%s","contacts":{"technical":"%s","support":"%s"},"status":"%s","sp1":{"wayflessUrl":"%s","checkTime":"%s","httpCode":"%s","checkResult":"%s"},"sp2":{"wayflessUrl":"%s","checkTime":"%s","httpCode":"%s","checkResult":"%s"}}\n' % (
getDisplayName(idp['displayname']), # IdP-DisplayName
idp['entityID'], # IdP-entityID
idp['registrationAuthority'], # IdP-RegAuth
str_technical_contacts, # IdP-TechCtcsList
str_support_contacts, # IdP-SuppCtcsList
idp_status, # IdP-ECCS-Status
check_results[0][1], # SP-wayfless-url-1
check_results[0][2], # SP-check-time-1
check_results[0][3], # SP-http-code-1
check_results[0][4], # SP-check-result-1
check_results[1][1], # SP-wayfless-url-2
check_results[1][2], # SP-check-time-2
check_results[1][3], # SP-http-code-2
check_results[1][4])) # SP-check-result-2
if (test):
sys.stdout.write("\nECCS2:")
sys.stdout.write('{"displayName":"%s","entityID":"%s","registrationAuthority":"%s","contacts":{"technical":"%s","support":"%s"},"status":"%s","sp1":{"wayflessUrl":"%s","checkTime":"%s","httpCode":"%s","checkResult":"%s"},"sp2":{"wayflessUrl":"%s","checkTime":"%s","httpCode":"%s","checkResult":"%s"}}\n' % (
get_display_name(idp['displayname']), # IdP-DisplayName
idp['entityID'], # IdP-entityID
idp['registrationAuthority'], # IdP-RegAuth
str_technical_contacts, # IdP-TechCtcsList
str_support_contacts, # IdP-SuppCtcsList
idp_status, # IdP-ECCS-Status
check_results[0][1], # SP-wayfless-url-1
check_results[0][2], # SP-check-time-1
check_results[0][3], # SP-http-code-1
check_results[0][4], # SP-check-result-1
check_results[1][1], # SP-wayfless-url-2
check_results[1][2], # SP-check-time-2
check_results[1][3], # SP-http-code-2
check_results[1][4])) # SP-check-result-2
else:
print("\nECCS2:")
print('{"displayName":"%s","entityID":"%s","registrationAuthority":"%s","contacts":{"technical":"%s","support":"%s"},"status":"%s","sp1":{"wayflessUrl":"%s","checkTime":"%s","httpCode":"%s","checkResult":"%s"},"sp2":{"wayflessUrl":"%s","checkTime":"%s","httpCode":"%s","checkResult":"%s"}}\n' % (
getDisplayName(idp['displayname']), # IdP-DisplayName
# IdP-DisplayName;IdP-entityID;IdP-RegAuth;IdP-tech-ctc-1,IdP-tech-ctc-2;IdP-supp-ctc-1,IdP-supp-ctc-2;IdP-ECCS-Status;SP-wayfless-url-1;SP-check-time-1;SP-http-code-1;SP-result-1;SP-wayfless-url-2;SP-check-time-2;SP-http-code-2;SP-result-2
with open(f"{e2p.ECCS2OUTPUTDIR}/{e2p.ECCS2RESULTSLOG}", 'a') as f:
try:
f.write('{"displayName":"%s","entityID":"%s","registrationAuthority":"%s","contacts":{"technical":"%s","support":"%s"},"status":"%s","sp1":{"wayflessUrl":"%s","checkTime":"%s","httpCode":"%s","checkResult":"%s"},"sp2":{"wayflessUrl":"%s","checkTime":"%s","httpCode":"%s","checkResult":"%s"}}\n' % (
get_display_name(idp['displayname']), # IdP-DisplayName
idp['entityID'], # IdP-entityID
idp['registrationAuthority'], # IdP-RegAuth
str_technical_contacts, # IdP-TechCtcsList
......@@ -261,45 +70,55 @@ def storeECCS2result(idp,check_results,idp_status,test):
check_results[1][1], # SP-wayfless-url-2
check_results[1][2], # SP-check-time-2
check_results[1][3], # SP-http-code-2
check_results[1][4])) # SP-check-result-2
check_results[1][4] # SP-check-result-2
)
)
except IOError:
sys.stderr.write(f"Failed writing result on output file for {idp['entityID']} with {utils.get_label(sp)}.\n\nRun {e2p.ECCS2DIR}/runEccs2.py --idp {idp['entityID']} --replace\n")
sys.exit(1)
# Check an IdP with 2 SPs.
def check(idp,sps,test):
def check(idp,test):
check_results = []
for sp in sps:
result = checkIdP(sp,idp,test)
if result is not None:
for sp in e2p.ECCS2SPS:
result = utils.check_idp_response_selenium(sp,idp,test)
if (result):
check_results.append(result)
if len(check_results) == 2:
else:
sys.stderr.write(f"\nCheck failed for {idp['entityID']} with {utils.get_label(sp)}.\n\nRun {e2p.ECCS2DIR}/runEccs2.py --idp {idp['entityID']} --replace\n")
sys.exit(1)
if (len(check_results) == len(e2p.ECCS2SPS)):
check_result_sp1 = check_results[0][4]
check_result_sp2 = check_results[1][4]
check_result_weberr1 = check_results[0][5]
check_result_weberr2 = check_results[1][5]
# If all checks are 'OK', than the IdP consuming correctly eduGAIN Metadata.
if (check_result_sp1 == check_result_sp2 == "OK"):
storeECCS2result(idp,check_results,'OK',test)
store_eccs_result(idp,sp,check_results,'OK',test)
elif (check_result_sp1 == check_result_sp2 == "DISABLED"):
storeECCS2result(idp,check_results,'DISABLED',test)
store_eccs_result(idp,sp,check_results,'DISABLED',test)
else:
storeECCS2result(idp,check_results,'ERROR',test)
store_eccs_result(idp,sp,check_results,'ERROR',test)
# MAIN
if __name__=="__main__":
sps = ECCS2SPS
parser = argparse.ArgumentParser(description='Checks if the input IdP consumed correctly eduGAIN metadata by accessing two different SPs')
parser.add_argument("idpJson", metavar="idpJson", nargs=1, help="An IdP in Json format")
parser.add_argument("--test", action='store_true', help="Test the IdP without effects")
parser.add_argument("--replace", action='store_true', help="Check an IdP and replace the result")
args = parser.parse_args()
idp = json.loads(args.idpJson[0])
Path("%s/%s" % (ECCS2HTMLDIR,DAY)).mkdir(parents=True, exist_ok=True) # Create dir needed to page_source content
Path(f"{e2p.ECCS2HTMLDIR}/{e2p.DAY}").mkdir(parents=True, exist_ok=True) # Create dir needed to page_source content
if (args.replace and not args.test):
utils.delete_line_with_word(f"{e2p.ECCS2OUTPUTDIR}/{e2p.ECCS2RESULTSLOG}",idp['entityID'])
check(idp,sps,args.test)
check(idp,args.test)
......@@ -22,9 +22,9 @@ ECCS2HTMLDIR = "%s/html" % ECCS2DIR
# Selenium
ECCS2SELENIUMDEBUG = False
ECCS2SELENIUMLOGDIR = "%s/selenium-logs" % ECCS2DIR
ECCS2SELENIUMPAGELOADTIMEOUT = 60 #seconds
ECCS2SELENIUMSCRIPTTIMEOUT = 60 #seconds
ECCS2REQUESTSTIMEOUT = 60 #seconds
ECCS2SELENIUMPAGELOADTIMEOUT = 30 #seconds
ECCS2SELENIUMSCRIPTTIMEOUT = 30 #seconds
ECCS2REQUESTSTIMEOUT = 15 #seconds
# Logs
ECCS2LOGSDIR = "%s/logs" % ECCS2DIR
......@@ -36,14 +36,22 @@ ECCS2STDERRIDP = "%s/stderr_idp_%s.log" % (ECCS2LOGSDIR,DAY)
ECCS2FAILEDCMDIDP = "%s/failed-cmd-idp.sh" % ECCS2LOGSDIR
# Number of processes to run in parallel
ECCS2NUMPROCESSES = 10
ECCS2NUMPROCESSES = 35
# The 2 SPs that will be used to test each IdP
#ECCS2SPS = ["https://sp24-test.garr.it/Shibboleth.sso/Login?entityID=", "https://attribute-viewer.aai.switch.ch/Shibboleth.sso/Login?entityID="]
ECCS2SPS = ["https://sp-demo.idem.garr.it/Shibboleth.sso/Login?entityID=", "https://attribute-viewer.aai.switch.ch/Shibboleth.sso/Login?entityID="]
ECCS2SPS = [
"https://sp-demo.idem.garr.it/Shibboleth.sso/Login?entityID=",
"https://attribute-viewer.aai.switch.ch/interfederation-test/Shibboleth.sso/Login?entityID="
]
# ROBOTS.TXT
ROBOTS_USER_AGENT = "ECCS/2.0 (+https://technical.edugain.org/eccs2)"
ROBOTS_USER_AGENT = "ECCS/2.0 (+https://technical-test.edugain.org/eccs2)"