Commit 8faa1196 authored by Marco Malavolti's avatar Marco Malavolti
Browse files

Fixed bugs

parent a6650dd9
......@@ -43,7 +43,7 @@ The check executed by the service follows these steps:
1. It retrieves the eduGAIN IdPs from eduGAIN Operator Team database via a JSON interface
2. For each IdP that is was not manually disabled by the eduGAIN Operations Team, the check creates a Wayfless URL for each SP involved and retrieves the IdP login page. It expects to find the HTML form with a username and password field. Therefore, no complete login will happen at the Identity Provider because the check stops at the login page.
The SPs used for the check are "Test SP shib 2.4" (https://sp-demo.idem.garr.it/shibboleth) from IDEM GARR AAI and the "AAI Viewer Interfederation Test" (https://attribute-viewer.aai.switch.ch/interfederation-test/shibboleth) from SWITCHaai. These SPs might change in the future if needed.
The SPs used for the check are "SP Demo provided by GARR" (https://sp-demo.idem.garr.it/shibboleth) from IDEM GARR AAI and the "AAI Viewer Interfederation Test" (https://attribute-viewer.aai.switch.ch/interfederation-test/shibboleth) from SWITCHaai. These SPs might change in the future if needed.
The SAML authenticatin request is not signed. Therefore, authentication request for any eduGAIN SP could be created because the SP's private key is not needed.
# Limitations
......@@ -70,9 +70,15 @@ The test eduGAIN Connectivity Check web pages is available at: https://technical
The tool uses following status for IdPs:
* ERROR (red):
* The IdP's response contains an HTTP Error or the web page returned does not look like a login page. The most probable causes for this error are HTTP errors (e.g.: 404 error)
* The IdP most likely does not consume the eduGAIN metadata correctly or it hasn't does not return a web page that looks like a login form. A typical case that falls into this category is when an IdP returns a message "No return endpoint available for relying party" or "No metadata found for relying party".
* The IdP has a problem with its SSL certificate.
* The IdP's response contains an HTTP Error or the web page returned does not look like a login page.
* **Invalid-Form**: considers those IdPs that do not load a standard username/password login page and do not return messages like "No return endpoint available for relying party" or "No metadata found for relying party".
* **Timeout**: considers those IdPs that do not load a standard username/password login page within 60 seconds.
* **Connection-Error**: considers those IdPs that are not reachable due to a connection problem. View the "Page Source" value to discover which problem the IdP has.
* The IdP most likely does not consume the eduGAIN metadata correctly.
A typical case that falls into this category is when an IdP returns a message "No return endpoint available for relying party" or "No metadata found for relying party":
* **No-eduGAIN-Metadata**
* The IdP has a problem with its SSL certificate:
* **SSL-Error**
* OK (green):
* The IdP most likely correctly consumes eduGAIN metadata and returns a valid login page. This is no guarantee that login on this IdP works for all eduGAIN services but if the check is passed for an IdP, this is probable.
* DISABLED (white)
......
......@@ -48,7 +48,7 @@ ECCS2SPS = [
ROBOTS_USER_AGENT = "ECCS/2.0 (+https://technical-test.edugain.org/eccs2)"
# PATTERNS
METADATAPATTERN = "Unable.to.locate(\sissuer.in|).metadata(\sfor|)|no.metadata.found|profile.is.not.configured.for.relying.party|Cannot.locate.entity|fail.to.load.unknown.provider|does.not.recognise.the.service|unable.to.load.provider|Nous.n'avons.pas.pu.(charg|charger).le.fournisseur.de service|Metadata.not.found|application.you.have.accessed.is.not.registered.for.use.with.this.service|Message.did.not.meet.security.requirements|Unsupported.Request|Not.Authorized"
METADATAPATTERN = "Unable.to.locate(\sissuer.in|).metadata(\sfor|)|no.metadata.found|profile.is.not.configured.for.relying.party|Cannot.locate.entity|fail.to.load.unknown.provider|does.not.recognise.the.service|unable.to.load.provider|Nous.n'avons.pas.pu.(charg|charger).le.fournisseur.de service|Metadata.not.found|application.you.have.accessed.is.not.registered.for.use.with.this.service|Message.did.not.meet.security.requirements|Unsupported.Request|Not.Authorized|METADATANOTFOUND|Unknown.login.requester|is.unspecified.or.unsupported|Unknown.service.provider|Richiesta.non.supportata|Metadati.non.trovati|untrusted.provider|Unregistered.Service"
USERNAMEPATTERN = '<input[\s]+[^>]*((type=\s*[\'"](text|email)[\'"]|user)|(name=\s*[\'"](name)[\'"]))[^>]*>'
PASSWORDPATTERN = '<input[\s]+[^>]*(type=\s*[\'"]password[\'"]|password)[^>]*>'
REFUSEDPATTERN = '(^http)(.*\.png$)|(.*\.css$)|(.*\.js$)|(.*\.gif$)|(.*\.svg$)|(.*\.jpg$)'
......
......@@ -11,9 +11,6 @@ import utils
from subprocess import PIPE
#from utils import get_list_feds, get_list_eccs_idps, get_reg_auth_dict, get_idp_list, gen_output
#from eccs2properties import ECCS2FAILEDCMD, ECCS2FAILEDCMDIDP, ECCS2STDOUT, ECCS2STDERR, ECCS2STDOUTIDP, ECCS2STDERRIDP, ECCS2DIR, ECCS2NUMPROCESSES, ECCS2LISTIDPSURL, ECCS2LISTIDPSFILE, ECCS2LISTFEDSURL, ECCS2LISTFEDSFILE, ECCS2OUTPUTDIR, ECCS2RESULTSLOG, ECCS2AUXDIR
# Run Command
# https://docs.python.org/3/library/asyncio-queue.html#examples
# https://docs.python.org/3/library/asyncio-subprocess.html
......@@ -31,7 +28,7 @@ async def run(name,queue,stdout_file,stderr_file,cmd_file):
stdout, stderr = await proc.communicate()
print(f'[{name} exited with {proc.returncode}] - {cmd!r}')
#print(f'[{name} exited with {proc.returncode}] - {cmd!r}')
if stdout:
stdout_file.write(f'-----\n[cmd]\n{cmd}\n\n[stdout]\n{stdout.decode()}')
......@@ -151,5 +148,4 @@ if __name__=="__main__":
cmd_file.close()
end = time.time()
#utils.gen_output(e2p.ECCS2AUXDIR,f"{e2p.ECCS2OUTPUTDIR}/{e2p.ECCS2RESULTSLOG}")
print("Time taken in hh:mm:ss - ", str(datetime.timedelta(seconds=end - start)))
......@@ -138,6 +138,7 @@ def get_idp_contacts(idp,contactType):
ctcList.append('missing email')
return ctcList
# Write the login page source code into its file
def store_page_source(page_source,idp,sp,test):
if (test):
......@@ -145,13 +146,14 @@ def store_page_source(page_source,idp,sp,test):
return True
else:
# Put the page_source into an appropriate HTML file
with open(f"{e2p.ECCS2HTMLDIR}/{e2p.DAY}/{sha1(['entityID'])}---{get_label(sp)}.html","w") as html:
with open(f"{e2p.ECCS2HTMLDIR}/{e2p.DAY}/{sha1(idp['entityID'])}---{get_label(sp)}.html","w") as html:
try:
html.write(page_source)
return True
except IOError:
return False
# Get the Google Chrom Selenium Driver
def get_driver_selenium(idp=None,sp=None,debugSelenium=False):
......@@ -188,6 +190,7 @@ def get_driver_selenium(idp=None,sp=None,debugSelenium=False):
driver = webdriver.Chrome(e2p.PATHCHROMEDRIVER, options=chrome_options)
return driver
# ECCS2 Check made by Selenium
def check_idp_response_selenium(sp,idp,test):
# Disable SSL requests warning messages
......@@ -230,13 +233,13 @@ def check_idp_response_selenium(sp,idp,test):
# Catch SSL Exceptions and block the ECCS check
except requests.exceptions.SSLError as e:
if (test): page_source = f"\nAn SSL Error occurred while opening https://{fqdn_idp}/robots.txt.\n\n{e}\n\nCheck it on SSL Labs: https://www.ssllabs.com/ssltest/analyze.html?d={fqdn_idp}"
if (test): page_source = f"\nAn SSL Error occurred while opening https://{fqdn_idp}/robots.txt:\n\n{e}\n\nCheck it on SSL Labs: https://www.ssllabs.com/ssltest/analyze.html?d={fqdn_idp}"
else: page_source = f"<h1>SSL ERROR</h1><h2>An SSL error occurred for the server {fqdn_idp}:</h2><p>{e}</p><p>Check it on SSL Labs: <a href='https://www.ssllabs.com/ssltest/analyze.html?d={fqdn_idp}'>Click Here</a></p>"
store_page_source(page_source,idp,sp,test)
return (idp['entityID'],wayfless_url,check_time,"(failed)","SSL-Error",webdriver_error)
except requests.exceptions.ConnectionError as e:
if (test): page_source = f"\nA Connection error occurred while opening https://{fqdn_idp}/robots.txt.\n\n{e}"
if (test): page_source = f"\nA Connection error occurred while opening https://{fqdn_idp}/robots.txt:\n\n{e}"
else: page_source = f"<h1>CONNECTION ERROR:</h1><h2>A Connection error occurred while opening <a href='https://{fqdn_idp}/robots.txt'>https://{fqdn_idp}/robots.txt</a>:</h2><p>{e}</p>"
store_page_source(page_source,idp,sp,test)
return (idp['entityID'],wayfless_url,check_time,"(failed)","Connection-Error",webdriver_error)
......@@ -248,7 +251,7 @@ def check_idp_response_selenium(sp,idp,test):
return (idp['entityID'],wayfless_url,check_time,"(failed)","Connection-Error",webdriver_error)
except requests.exceptions.TooManyRedirects as e:
if (test): page_source = f"\nToo many redirects occurred while opening: https://{fqdn_idp}/robots.txt.\n\n{e}"
if (test): page_source = f"\nToo many redirects occurred while opening: https://{fqdn_idp}/robots.txt:\n\n{e}"
else: page_source = f"<h1>TOO MANY REDIRECTS</h1><h2>Too many redirects occurred while opening: <a href='https://{fqdn_idp}/robots.txt'>https://{fqdn_idp}/robots.txt</a>:</h2><p>{e}</p>"
store_page_source(page_source,idp,sp,test)
return (idp['entityID'],wayfless_url,check_time,"(failed)","Connection-Error",webdriver_error)
......@@ -299,21 +302,27 @@ def check_idp_response_selenium(sp,idp,test):
stored = store_page_source(pgsrc,idp,sp,test)
if (stored):
return (idp['entityID'],wayfless_url,check_time,http_code,"No-eduGAIN-Metadata",webdriver_error)
else:
if (test): pgsrc = f"Timeout: No access form found in {e2p.ECCS2SELENIUMPAGELOADTIMEOUT} seconds"
elif(page_source == "<html><head></head><body></body></html>"):
if (test): pgsrc = f"Timeout: No valid login form loaded in {e2p.ECCS2SELENIUMPAGELOADTIMEOUT} seconds"
else: pgsrc = page_source
stored = store_page_source(pgsrc,idp,sp,test)
if (stored):
return (idp['entityID'],wayfless_url,check_time,"(failed)","Timeout",webdriver_error)
else:
if (test): pgsrc = f"Invalid-Form: No valid login form found in {e2p.ECCS2SELENIUMPAGELOADTIMEOUT} seconds"
else: pgsrc = page_source
stored = store_page_source(pgsrc,idp,sp,test)
if (stored):
return (idp['entityID'],wayfless_url,check_time,"(failed)","Invalid-Form",webdriver_error)
except WebDriverException as e:
error = e.__dict__['msg'].split('(')[0].rstrip()
if (test): pgsrc = f"\nERROR: {error}"
else: pgsrc = f"<h1>ECCS2 CHECK FAILED</h1><h2>The IdP Login failed the check due the following error:</h2><p>{error}</p>"
if (test): pgsrc = f"\nA Connection error occurred while opening {wayfless_url}:\n\n{error}"
else: pgsrc = f"<h1>CONNECTION ERROR</h1><h2>A Connection error occurred while opening <a href='{wayfless_url}'>{wayfless_url}</a>:</h2><p>{error}</p>"
webdriver_error = 1
stored = store_page_source(pgsrc,idp,sp,test)
if (stored):
return (idp['entityID'],wayfless_url,check_time,"(failed)","ERROR",webdriver_error)
return (idp['entityID'],wayfless_url,check_time,"(failed)","Connection-Error",webdriver_error)
finally:
driver.quit()
......
......@@ -193,10 +193,10 @@ function getCheckResult(checkResult){
return '<div class="tooltip">OK <span class="tooltiptext tooltip-top tooltip-ok">The IdP is consuming correctly the eduGAIN metadata and return a valid login page</span></div> '+infoCircle;
}
else if (checkResult == "Timeout"){
return '<div class="tooltip">Timeout <span class="tooltiptext tooltip-top tooltip-timeout">The IdP does not load a valid login page within 60 seconds</span></div> '+infoCircle;
return '<div class="tooltip">Timeout <span class="tooltiptext tooltip-top tooltip-timeout">The IdP does not load a valid login page within 30 seconds</span></div> '+infoCircle;
}
else if (checkResult == "Invalid-Form"){
return '<div class="tooltip">Invalid-Form <span class="tooltiptext tooltip-top tooltip-invalid-form">The IdP does not load a valid login page</span></div> '+infoCircle;
else if (checkResult == "Connection-Error"){
return '<div class="tooltip">Connection Error <span class="tooltiptext tooltip-top tooltip-invalid-form">Check failed due a connection error</span></div> '+infoCircle;
}
else if (checkResult == "No-eduGAIN-Metadata"){
return '<div class="tooltip">No-eduGAIN-Metadata <span class="tooltiptext tooltip-top tooltip-no-edugain-metadata">The IdP is not consuming correctly edugGAIN metadata stream</span></div> '+infoCircle
......@@ -229,7 +229,7 @@ function format ( d ) {
'<td>'+d.contacts.support+'</td>'+
'<td class="strong">Check Time</td>'+
'<td class="strong">Check Result</td>'+
'<td class="strong">HTTP Code</td>'+
//'<td class="strong">HTTP Code</td>'+
'<td class="strong">Page Source</td>'+
'<td class="strong">Retry Check</td>'+
'</tr>'+
......@@ -238,7 +238,7 @@ function format ( d ) {
'<td>https://'+getHostname(d.sp1.wayflessUrl)+'</td>'+
'<td>'+d.sp1.checkTime+'</td>'+
'<td>'+getCheckResult(d.sp1.checkResult)+'</td>'+
'<td>'+d.sp1.httpCode+'</td>'+
//'<td>'+d.sp1.httpCode+'</td>'+
'<td><a href="/eccs2html/'+d.date+'/'+SHA1(d.entityID)+'---'+getHostname(d.sp1.wayflessUrl)+'.html" target="_blank">Click to open</a></td>'+
'<td><a href="'+d.sp1.wayflessUrl+'" target="_blank">Click to retry</a></td>'+
'</tr>'+
......@@ -247,7 +247,7 @@ function format ( d ) {
'<td>https://'+getHostname(d.sp2.wayflessUrl)+'</td>'+
'<td>'+d.sp2.checkTime+'</td>'+
'<td>'+getCheckResult(d.sp2.checkResult)+'</td>'+
'<td>'+d.sp2.httpCode+'</td>'+
//'<td>'+d.sp2.httpCode+'</td>'+
'<td><a href="/eccs2html/'+d.date+'/'+SHA1(d.entityID)+'---'+getHostname(d.sp2.wayflessUrl)+'.html" target="_blank">Click to open</a></td>'+
'<td><a href="'+d.sp2.wayflessUrl+'" target="_blank">Click to retry</a></td>'+
'</tr>'+
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment