Handling SSL Certificates


How to handle SSL Certificates ..??

This Exception may be common to all , but my fingers crossed when I see the next wizard, as shown in second image, 

Image 1:  This Connection is Untrusted
If we click 'Add Exception' button in the first page, then the next wizard comes up as shown below

Image 2: Add Security Exception

After a long search on some forums, I found that a small piece of code are not explained by most of the them or may be I come across most of such blogs. However, I thought of to put what I have discovered and handled this issue and where I was missing or may be you are missing the actual explanation in other blogs

Apparently, handling this ssl certificate can be achieved on firefox browser only, because it allows users to access the profile and where as other browsers doesn't have option for creating custom browser profile, if any one knows, please let me know in comments

Now, before explaining code, I want to explain how to create custom firefox profile then the code might be easily understands at its a very few in lines. 

How to create custom firefox profile?

1) Close all firefox browser windows and run the following command
firefox -ProfileManager -no-remote
2) Click "Create Profile"  and follow the instructions on the wizard
3) You can create with any name, as I created with "QAAutomation" as shown in below image

"firefox -ProfileManager -no-remote" to choose firefox profile

This is how we can create profiles in firefox browser, now we have to use this profile in our selenium webdirver code, as follows

#Java Program:

Method 1:
/**
 * Handle Untrusted HTTPs Connections With Selenium web driver. Also, handle an
 * additional wizards like, 'Add Security Exception' by Confirm Security
 * Exception
 * 
 * @author Fayaz
 * 
 */
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.openqa.selenium.firefox.FirefoxProfile;
import org.openqa.selenium.firefox.internal.ProfilesIni;


public class HandleWindows {

public static void main(String[] args){
ProfilesIni profile = new ProfilesIni();
FirefoxProfile ffProfile = profile.getProfile("QAAutomation"); // firefox profile **
ffProfile.setAcceptUntrustedCertificates(true);
ffProfile.setAssumeUntrustedCertificateIssuer(false);
WebDriver driver = new FirefoxDriver(ffProfile);
driver.get(URL);
}
}


** Most of the blogs are pasted code as it is, where some says, "HandleSSLCertificate" but they don't explain what is this "HandleSSLCertificate", whereas others explains simply as "firefox user profile name" (but they missed how to create profile, indeed)


Remember most of us use "default" as a profile name, by default. So please pass "default" as an parameter to getProfile() and try if you want, that is new ProfilesIni().getProfile("default");

Method 2:

/**
 * Handle Untrusted HTTPs Connections With Selenium web driver. Also, handle an
 * additional wizards like, 'Add Security Exception' by Confirm Security
 * Exception
 *
 * @author Fayaz
 *
 */
import java.io.File;

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.openqa.selenium.firefox.FirefoxProfile;
import org.openqa.selenium.firefox.internal.ProfilesIni;


public class HandleWindows {

 public static void main(String[] args){
 FirefoxProfile ffProfile = new FirefoxProfile(new File("C:\\Users\\Fayaz\\Desktop\\Selenium\\custom")); // firefox profile**
  ffProfile.setAcceptUntrustedCertificates(true);
  ffProfile.setAssumeUntrustedCertificateIssuer(false);
  WebDriver driver = new FirefoxDriver(ffProfile);
  driver.get(URL);

 }
}

** While creating a firefox profile I created a new folder with name 'custom'. That is though "QAAutomation" is actual firefox profile name, it is created under 'custom' directory. so custom is a my favorite directory on my machine where as QAAutomation is a firefox profile name


Finally, those who wants to execute test from command line interface aka CLI, then the command should be like, 
java -jar selenium-server.jar -firefoxProfileTemplate "C:\Users\Fayaz\Desktop\Selenium",
instead  of the giving the whole default path like, 
java -jar selenium-server.jar -firefoxProfileTemplate "C:\Documents and Settings\Fayaz\Application Data\Mozilla\Firefox\Profiles\io4kgk8d.default"
That's all we are done. For more info see Tips & Tricks page

Chrome :

Though by the time of writing this blog page, I didn't find the similar way, how we are handling dialog firefox browser. However, one way I used to handle this certificate dialog while using chrome is using Threads in java

Before, you read further, let me tell you what my issue was

When I request to an URL by calling our selenium WebDrivers, get(String URL) method and before the response gets back, certificate dialog is appearing and until I select the certificate the URL page isn't loading.
System.setProperty("webdriver.chrome.driver", "C:\\Users\\Fayaz\\Desktop\\chromedriver.exe");
WebDriver driver = new ChromeDriver(); 
driver.get(URL); // this method execution is not finishing until I select a dialog in the //certificate dialog
driver.findElements(By.cssSelector("input#username")); //this code is not at all reaching

 So, inorder to handle this, what I did was, I was calling a Thread in which using Robot class, making enter key to press. That is, start a thread and make it to sleep for a while then do request to URL from driver.get() method, by the time certificate dialog appears our Thread process should finishes its sleep time and then start the thread to run, where in run() method, our Robot class hit the enter button

Here is the working code for me, how it works....        

#Java Program:

package selenium.project;

import java.awt.AWTException;
import java.awt.Robot;
import java.awt.event.KeyEvent;

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeOptions;
import org.openqa.selenium.remote.CapabilityType;
import org.openqa.selenium.remote.DesiredCapabilities;

/**
 * Handle Untrusted HTTPs Connections With Selenium web driver. Also, handle an
 * additional wizards like, 'Add Security Exception' by Confirm Security
 * Exception
 *
 * @author Fayaz
 *
 */
public class HandleWindows {

private static final String URL = "<your URL here> ";

public static void main(String[] args) {
WebDriver driver = null;
try {
System.setProperty("webdriver.chrome.driver", "C:\\Users\\Fayaz\\Downloads\\chromedriver.exe");
driver = new ChromeDriver();
Thread certSelectionThread = null;
Runnable r = new Runnable() {

@Override
public void run() {
try {
Thread.sleep(1000 * 10);
Robot robot = new Robot();
robot.keyPress(KeyEvent.VK_ENTER);
} catch (AWTException e) {
e.printStackTrace();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
};
certSelectionThread = new Thread(r);
certSelectionThread.start();
driver.get(URL);
if(certSelectionThread != null){
try {
certSelectionThread.join();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
} finally {
if(driver != null && !driver.toString().contains("null")){
driver.close();
driver.quit();
}
}
}
}



Sometime we need to handle certificate issues (SSL) with chrome as well especially for self-signed certificates. else we get exception, which says as follows

javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed:
sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:387)
at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292)
at sun.security.validator.Validator.validate(Validator.java:260)
at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:324)
at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:229)
at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:124)
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1351)
at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:156)
at sun.security.ssl.Handshaker.processLoop(Handshaker.java:925)
at sun.security.ssl.Handshaker.process_record(Handshaker.java:860)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1043)
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1343)
at sun.security.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:728)
at sun.security.ssl.AppOutputStream.write(AppOutputStream.java:123)
at sun.security.ssl.AppOutputStream.write(AppOutputStream.java:138)



when we run the following piece of code

public static boolean waitUntilRequestCompletes(String urlRequest) {
try {
URL url = new URL(urlRequest);
HttpURLConnection conn;
conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("GET");
conn.setRequestProperty("Accept", "application/json");

if (conn.getResponseCode() != 200) {
throw new RuntimeException(" HTTP error code : "
+ conn.getResponseCode());
}

Scanner scan = new Scanner(url.openStream());
String entireResponse = new String();
while (scan.hasNext()) {
entireResponse += scan.nextLine();
System.out.println("Response : " + entireResponse);
}
scan.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
return false;
}


To avoid such exception it is important to add signed certificate to JVM - security\cacerts.

How to do that.?

So from the insecure URL download the certificate, follow this page to know how to download certificate

Once the certificate in place, open command prompt in admin mode and run the following command

keytool -import -alias example_name -keystore "C:\Program Files\Java\jdk1.8.0_121\jre\lib\security\cacerts" -file "C:\Users\Fayaz\Desktop\ss_certificate\pd-123456789.cer"

That's it done. happy secure connection :)

Selenium Grid

Selenium Grid:

Well, I have experienced the following scenarios while setting up the grid environment for the first time. 

Here it is, I have two machine, say '10.19.137.247' (test machine) and 192.168.1.113 (test dev machine) so to start the hub on test machine I have run the following command:

On Test Machine:

    java -jar selenium-server-standalone.jar -role hub
The following message or information appeared on command line as a result
Mar 10, 2014 2:57:03 AM org.openqa.grid.selenium.GridLauncher main
INFO: Launching a selenium grid server
2014-03-10 02:57:26.332:INFO:osjs.Server:jetty-7.x.y-SANPSHOT
2014-03-10 02:57:26.395:INFO:osjsh.ContextHandler:started o.s.j.s.ServletContextHandler{/,null}
2014-03-10 02:57:26.410:INFO:osjs.AbstractConnector:Started SocketConnector@0.0.0.0:4444     

On Test Dev machine:

 java -jar selenium-server-standalone -Dwebdriver.chrome.driver=C:\Users\Fayaz\Downloads\chromedriver.exe -role webdriver -hub http://10.19.137.247/grid/register
The following message or information appeared on command line as a result
Mar 10,  2014 12:30:36 PM org.openqa.grid.selenium.GridLauncher main
INFO: Launching a selenium grid node
Mar 10, 2014 12:30:39 PM org.openqa.grid.internal.utils.SelfRegisteringRemote startRemoteServer
WARNING: error getting the parameters from the hub. The node may end up with wrong timeouts. Connection refused: connect
12:30:39.255 INFO - Java: Oracle Corportation 24.45-b08
12:30:39.263 INFO - OS:Windows 7 6.1 x86
12:30:39.295 INFO - v2.39.0, with Core v2.39.0. Built from revision ff23eac
12:30:39.565 INFO - Default driver org.openqa.selenium.iphone.IPhoneDriver registration is skipped: registration capabilities Capabilites [{platform=MAC, browserName=iPhone, version=}] does not match with current platform: VISTA
12:30:39:565 INFO - Default driver org.openqa.selenium.iphone.IPhoneDriver registration is skipped:
registration capabilities Capabilities [{platform=MAC, browserName=iPad, version=}] does not match with current platform: VISTA
12:30:39.648 INFO - RemoteWebDriver instances should connect to : http://127.0.0.1:5555/wd/hub
12:30:39.649 INFO - Version Jetty/5.1.x
12:30:39.650 INFO - Started HttpContext[/selenium-server/driver./selenium-server/driver]
12:30:39.651 INFO - Started HttpContext[/selenium-server,/selenium-server]
12:30:39:652 INFO - Started HttpContext[/,/]
12:30:39:654 INFO - Started org.openqa.jetty.jetty.servlet.ServletHandler@12132a6
12:30:39.654 INFO - Started HttpContext[/wd,/wd]
12:30:39.657 INFO - Started SocketListener on 0.0.0.0:5555
12:30:39.658 INFO - Started Started org.openqa.jetty.jetty.Server@a4b9da
12:30:39.661 INFO - using the json request : {"class":"org.openqa.grid.common.RegistrationRequest","capabilities":[{"platform":"VISTA","seleniumProtocol":"Selenium","browserName":"*firefox","maxInstances":5},{"platform":"VISTA","seleniumProtocol":"Selenium","browserName":"*googlechrome","maxInstances":5},{"platform":"VISTA","seleniumProtocol":"seleniumProtocol":"WebDriver","browserName":"chrome","maxInstances":1},{"platform":"VISTA","seleniumProtocol":"WebDriver","browserName":"internet explorer","maxInstances":1}],"configuration":{"port":5555,"register":true,"host":"192.168.1.113","proxy":"org.openqa.grid.selenium.proxy.DefaultRemoteProxy","maxSession":5,"role":"node","hubHost":"10.19.137.247","registerCycle":5000,"hub":"http://10.19.137.247/grid/register","hubPort":-1,"url":"http://192.168.1.113:5555","remoteHost":"http://192.168.1.113:5555"}}
12:30:39.664 INFO - Starting auto register thread. Will try to register every 5000 ms.
12:30:39.664 INFO - Registering the node to hub :http://10.19.137.247:-1/grid/register
12:30:41.466 INFO - couldn't register this node : Error sending the registration request.
12:30:48.264 INFO - couldn't register this node : Hub is down or not responding: Connection refused: connect
If we observer on the test machine to verify the status of the selenium web server whether the client / node is connected or not, in other words when no node connects to hub using selenium grid, it will looks like in the screen shot

When no nodes connects to a hub using selenium grid then the server details displayed


This error has been resolved after I corrected the command to connect the hub from my test dev machine, and the changed command is
java -jar selenium-server-standalone.jar -Dwebdriver.chrome.driver=C:\Users\Fayaz\Downloads\chromedriver.exe -role webdriver -hub http://10.19.137.247:4444/grid/register

The difference here is I am using port number ':4444' after test machine URL

After connecting to the server, we can verify the status of the selenium web server whether the client / node is connected or not, it will looks like in the screen shot


When a node connects to hub using selenium grid then the list of details displayed


The path in browser to see list of clients or nodes connected to a hub at server / hub side is,
 localhost:4444/grid/console
For a beginner most common and exception can occur as follows, part of exception pasted
Caused by: java.lang.IllegalStateException: The path to the driver executable must be set by the webdriver.chrome.driver system property;
For more info please find the source here

To handle ssl or certification errors on a remote browser, I have used the following code and it worked like a charm

For IE or Chrome:
  remoteWebdriver.navigate().to("javascript:document.getElementById('overridelink').click()");
//I assume you have already had a wed driver reference called 'remoteWebdriver'


Now, to take screen shot of the remote test script execution, you can find the code in this selenium webdriver cheat code or from here

Update :


From the latest release of selenium 3.x, if we use above command to register client to remote hub then we get the following exception

Exception in thread "main" com.beust.jcommander.ParameterException: Unknown option: -Dwebdriver.chrome.driver=C:\Users\Fayaz\Downloads\chromedriver.exe
        at com.beust.jcommander.JCommander.parseValues(JCommander.java:742)
        at com.beust.jcommander.JCommander.parse(JCommander.java:282)
        at com.beust.jcommander.JCommander.parse(JCommander.java:265)
        at com.beust.jcommander.JCommander.<init>(JCommander.java:210)
        at org.openqa.grid.selenium.GridLauncherV3$3.setConfiguration(GridLauncherV3.java:267)
        at org.openqa.grid.selenium.GridLauncherV3.buildLauncher(GridLauncherV3.java:155)
        at org.openqa.grid.selenium.GridLauncherV3.main(GridLauncherV3.java:75)




So the command to register a node / client is changed as follows:

                        java -jar selenium-server-standalone-3.3.1.jar -role node -hub http://<hub_ip>:4444/grid/register -port 4444

Browser Functions

Handling Browser Functionality

Scroll Down:

/**
 * Scroll to the bottom of a page and sleep for 10 seconds
 * 
 */
public static void scrollDown() {
((JavascriptExecutor) SeleniumAction.getDriver())
.executeScript("scroll(0, 250)");
sleep(10); // Customized sleep method
}

Scroll Up:

/**
 * Scroll to the top of a page and sleep for 10 seconds
 * 
 */
public static void scrollUp() {
((JavascriptExecutor) SeleniumAction.getDriver())
.executeScript("scroll(250, 0)");
sleep(10); // Customized sleep method
}


Scroll Upto an Element:

/**
 * Scroll to an web element to make it display on screen 
 *  
 * @param element
 */
public static void scrollToAnElement(WebElement element) {
if (isElementExists(element)) {
((Locatable) element).getCoordinates().inViewPort();
sleep(5); // Customized sleep method
}

}

Maximize a web browser:
/**
* Maximize a browser window
*/
public void maximizeBrowserWindow(){
       driver.manage().window().maximize();
}

Refresh a browser:
/**
* Refresh a browser window
*/
//Method #1:
public void refreshBrowserWindow(){
       driver.navigate().refresh();
}

//Method #2:
public void refreshBrowserWindow(){
       driver.get(driver.getCurrentUrl());
}

// Method #3:
public void refreshBrowserWindow(){
      driver.navigate().to(driver.getCurrentUrl());  
}

//Method #4:
public void refreshBrowserWindow(WebDriver driver){
      Actions refreshAction = new Actions(driver);
      refreshAction.keyDown(Keys.CONTROL).sendKeys(Keys.F5).perform();
}

Handling an alert:
/**
 * Handling an alert dialog 
 * 
 *@param discard
 */
public void handleAlert(boolean discard){
     Alert alertHandler =    driver.switchTo().alert();
     if(discard){
          alertHandler.dismiss();
     } else {
           alertHandler.accept();
     }
}

Take a screenshot:

/**
 * Takes a screen shot and save with the given file name
 *  
 * @param fileName
 * @throws IOException
 */

public static void takeScreenshot(String fileName) throws IOException {
File screenshot = ((TakesScreenshot) SeleniumAction.getDriver())
.getScreenshotAs(OutputType.FILE);
String dirPath = ScriptsExecuted.getRFTLogDirectory()
+ fileName
+ new SimpleDateFormat("MM-dd-yyy_HH:ss")
.format(new GregorianCalendar().getTime() + ".jpg");
FileUtils.moveFile(screenshot, new File(dirPath));
}

// For taking a screen shot on remote web driver, source

/**
  *  RemoteWebDriver does not implement the TakesScreenshot class
  *  if the driver does have the Capabilities to take a screenshot

  *  then Augmenter will add the TakesScreenshot methods to the instance
  */
public void myTest() throws Exception {
        WebDriver driver = new RemoteWebDriver(
                                new URL("http://localhost:4444/wd/hub"), 
                                DesiredCapabilities.firefox());
        
        driver.get("http://www.google.com");
        WebDriver augmentedDriver = new Augmenter().augment(driver);
        File screenshot = ((TakesScreenshot)augmentedDriver).
                            getScreenshotAs(OutputType.FILE);
}


Verification of an element existence:

/**
 * Method to verify whether desired web element is exists or not by handling
 * exceptions
 * 
 * @param element
 * @return
 */
public static boolean isElementExists(WebElement element) {
boolean result = false;
try {
if (element != null) {
result = element.isDisplayed();
}
} catch (NoSuchElementException noElementExecption) {
result = false;
} catch (StaleElementReferenceException staleElementException) {
return false;
}
return result;
}


Wait For page to load

void waitForLoad(WebDriver driver) {
    ExpectedCondition<Boolean> pageLoadCondition = new
        ExpectedCondition<Boolean>() {
            public Boolean apply(WebDriver driver) {
                return ((JavascriptExecutor)driver).executeScript("return document.readyState").equals("complete");
            }
        };
    WebDriverWait wait = new WebDriverWait(driver, 30);
    wait.until(pageLoadCondition);

}


Validate Image Loaded properly

/**
  * Import statements, 
  */
import org.apache.http.HttpResponse;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.HttpClientBuilder;

public void validateInvalidImages() {
try {
invalidImageCount = 0;
List<WebElement> imagesList = driver.findElements(By.tagName("img"));
System.out.println("Total no. of images are " + imagesList.size());
for (WebElement imgElement : imagesList) {
if (imgElement != null) {
verifyimageActive(imgElement);
}
}
System.out.println("Total no. of invalid images are " + invalidImageCount);
} catch (Exception e) {
e.printStackTrace();
System.out.println(e.getMessage());
}
}

public void verifyimageActive(WebElement imgElement) {
try {
HttpClient client = HttpClientBuilder.create().build();
HttpGet request = new HttpGet(imgElement.getAttribute("src"));
HttpResponse response = client.execute(request);
// verifying response code he HttpStatus should be 200 if not,
// increment as invalid images count
if (response.getStatusLine().getStatusCode() != 200)
invalidImageCount++;
} catch (Exception e) {
e.printStackTrace();
}

}