[FAQ] Verifying the Host Certificate in Script with CA certificate / custom CA

Advertisement

Makc666
Joined:
Posts:
50
Location:
MSK-RU

[FAQ] Verifying the Host Certificate in Script with CA certificate / custom CA

For now "Verifying the Host Key or Certificate in Script" is done with -hostkey="<fingerprint>" or -certificate="<fingerprint>"

Like:
open ftps://martin:mypassword@example.com/ -implicit -certificate="xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx"
And this makes WinSCP automatically accept certificate with the fingerprint.

But if fingerprint will change the script will stop working until you will update the fingerprint.

Can you implement Verifying the Host Certificate in Script with CA certificate?

As you know any certificate is issued by CA (Certificate Authority).
And if you have that CA certificate you can check with it that server's certificate was signed by that CA (some one call is Issuer CA or SubCA or Root CA).
In any way there is usually a chain like "Certificate <- Issuer CA <- Root CA".

Am asking to implement a way so we can give WinSCP "Issuer CA <- Root CA" certificates and ask WinSCP to check server's certificate when we connect to that server.

This will be very handful as will allow the script to work if the certificate will be changed / issued by the trusted CA.

Thanks!
Last edited by Makc666 on 2017-02-14 13:19; edited 2 times in total

Reply with quote

Advertisement

martin
Site Admin
martin avatar
Joined:
Posts:
41,454
Location:
Prague, Czechia

Re: Verifying the Host Certificate in Script with CA certificate

WinSCP does verify the certificate against certificate roots in Windows certificate store.
WinSCP session log records the process.

You need to use the -certificate switch only if the certificate is not signed by a trusted root certificate or if it is a self-signed certificate.

Reply with quote

Makc666
Joined:
Posts:
50
Location:
MSK-RU

Re: Verifying the Host Certificate in Script with CA certificate

@martin,
if I will add my own ROOT certificate (my own CA by which my self-signed certificate was signed) to Windows certificate store, then it will work?!

I wasn't able to do so.

I added my own Root CA and SubCA to Windows storage (see screenshots):
Certificates - Local Computer
- Trusted Root Certification Authorities - Certificates
- Intermediate Certification Authorities - Certificates

Then I remove -certificate="*" from command line which is run from /script=Script.txt

Tried to connect to the Server which has certificate which was signed by my own CA.

And got error - see log attached (private).

Here is cut from it:
. 2017-02-01 15:55:49.760 --------------------------------------------------------------------------
. 2017-02-01 15:55:49.766 Connecting to 123.123.123.123:443 ...
. 2017-02-01 15:55:49.942 Connected with 123.123.123.123:443, negotiating TLS connection...
< 2017-02-01 15:55:50.458 220- DataExpress FTP Server ready.
< 2017-02-01 15:55:50.458 220 comForte SecurFTP version SERVER_NAME in FTP server mode
> 2017-02-01 15:55:50.458 AUTH TLS
< 2017-02-01 15:55:50.656 234 SecurFTP: SSL starting
. 2017-02-01 15:55:51.290 Server asks for authentication with a client certificate.
. 2017-02-01 15:55:51.750 Verifying certificate for "ORGANIZATION_NAME" with fingerprint 11:22:33:44:55:66:77:88:99:00:11:22:33:44:55:66:77:88:99:00 and 19 failures
. 2017-02-01 15:55:51.751 Asking user:
. 2017-02-01 15:55:51.751 **The server's certificate is not known. You have no guarantee that the server is the computer you think it is.**
. 2017-02-01 15:55:51.751 
. 2017-02-01 15:55:51.751 Server's certificate details follow:
. 2017-02-01 15:55:51.751 
. 2017-02-01 15:55:51.751 Issuer:
. 2017-02-01 15:55:51.751 - Organization: Delivery External CA
. 2017-02-01 15:55:51.751 - Location: US
. 2017-02-01 15:55:51.751 
. 2017-02-01 15:55:51.751 Subject:
. 2017-02-01 15:55:51.751 - Organization: ORGANIZATION_NAME
. 2017-02-01 15:55:51.751 - Location: US, City
. 2017-02-01 15:55:51.751 
. 2017-02-01 15:55:51.751 Valid: 25.03.2015 21:26:30 - 25.03.2018 21:26:30
. 2017-02-01 15:55:51.751 
. 2017-02-01 15:55:51.751 Fingerprint (SHA-1): 11:22:33:44:55:66:77:88:99:00:11:22:33:44:55:66:77:88:99:00
. 2017-02-01 15:55:51.751 
. 2017-02-01 15:55:51.751 Summary: Self-signed certificate in certificate chain. The error occurred at a depth of 3 in the certificate chain.
. 2017-02-01 15:55:51.751 
. 2017-02-01 15:55:51.751 When connecting using an IP address, it is not possible to verify if the certificate was issued for the server. Use a hostname instead of the IP address.
. 2017-02-01 15:55:51.751 
. 2017-02-01 15:55:51.751 If you trust this certificate, press Yes. To connect without storing certificate, press No. To abandon the connection press Cancel.
. 2017-02-01 15:55:51.751 
. 2017-02-01 15:55:51.751 Continue connecting and store the certificate? ()
. 2017-02-01 15:56:01.952 Peer certificate rejected
. 2017-02-01 15:56:01.952 Disconnected from server
. 2017-02-01 15:56:01.952 Connection failed.

I have to add remote server's certificate to Windows storage also? Then to which part of the storage?

Thank you!

winscp-net_forum_t-24406_02.png

winscp-net_forum_t-24406_01.png

  • winscp-net_forum_t-24406_01.log (6.02 KB, Private file)

Reply with quote

martin
Site Admin
martin avatar

Re: Verifying the Host Certificate in Script with CA certificate

I've never done this myself. So I cannot provide you more details.

But you definitely does not have to add the certificate itself to the certificate store.

Reply with quote

Makc666
Joined:
Posts:
50
Location:
MSK-RU

Re: Verifying the Host Certificate in Script with CA certificate

@martin, I found the solution.

I think you have to Document this solution because really it is build into WinSCP but not documented.

I found solution here: https://www.rootusers.com/problems-with-winscp-verifying-ssl-certificates/
I will quote the valuable part of the post at the end of my message.

Solution

1. WinSCP uses file cacert.pem from the same folder where winscp.exe is placed.

2. You have to put into that file cacert.pem your commercial or self-signed CA-ROOT and SUB-CA certificates.

3. You have to connect to the host using host name (not IP-address).

If all points above are done correctly you don't have to use -certificate="<fingerprint>" for FTPS.

That it. Really very easy in this case.

My question to @martin

after all investigations I expect that WinSCP does NOT use Windows Certificate Storage.
If the source code from the above link is actual then is it really that cacert.pem file is the only place where WinSCP is looking for trusted CA certificates?
Can you please confirm that WinSCP doesn't look into Windows Certificate Storage?

If cacert.pem is the only way may be you can make this future more flexible so it can be setup throw interface?
But in any way I think that the current way of work has to be documented. :)

P.S.
The below one is valuable part of the post from:
https://www.rootusers.com/problems-with-winscp-verifying-ssl-certificates/
Lately I’ve been working on setting up a file server on an Amazon ec2 instance with ProFTPD so that I can upload files using FTPS/SFTP. I had some issues getting FTPS to work correctly despite using a Rapid SSL certificate from GeoTrust that I purchased. I attempted first with FileZilla, and then with WinSCP but both gave me the below error and had problems verifying my SSL certificate.

Unable to get local issuer certificate. The error occurred at a depth of 2 in the certificate chain.

...here some text is cut out...

By looking into the WinSCP code we can check exactly how it is verifying the SSL certificate:
filezilla/AsyncSslSocketLayer.cpp:
      if (clientMode)
      {
         USES_CONVERSION;
         pSSL_CTX_set_verify(m_ssl_ctx, SSL_VERIFY_PEER, verify_callback);
         pSSL_CTX_load_verify_locations(m_ssl_ctx, T2CA(m_CertStorage), 0);
      }
filezilla/AsyncSslSocketLayer.cpp:
BOOL CAsyncSslSocketLayer::SetCertStorage(CString file)
{
   m_CertStorage = file;
   return TRUE;
}
filezilla/FtpControlSocket.cpp:
      m_pSslLayer = new CAsyncSslSocketLayer;
      AddLayer(m_pSslLayer);
      TCHAR buffer[1000];
      GetModuleFileName(NULL, buffer, 1000);
      CString filename = buffer;
      int pos = filename.ReverseFind(_MPT('\\'));
      if (pos != -1)
      {
         filename = filename.Left(pos + 1);
         filename += _T("cacert.pem");
         m_pSslLayer->SetCertStorage(filename);
      }
      else
         filename = _MPT("cacert.pem");
So WinSCP uses OpenSSL to verify certificates, the function pSSL_CTX_load_verify_locations specifies the location of trusted CA certificates. In this instance we can see that it uses a file called cacert.pem in the same location as the running module.

Basically WinSCP does not come with pre-trusted certificates, which is preferable for security reasons but a bit inconvenient as I couldn’t find this information documented. The only information I could find was this which implied that the certificates from the Windows store would be used, and I definitely had the GeoTrust root certificate there.

To fix this I added all the Firefox CAs from https://curl.se/docs/caextract.html

Just download cacert.pem from the above link and place it into the same folder as winscp.exe, then when you try to connect to the server with FTPS that has your certificate you shouldn’t get any certificate warnings.
Thanks!
Last edited by Makc666 on 2017-02-06 08:48; edited 1 time in total

Reply with quote

Advertisement

martin
Site Admin
martin avatar
Joined:
Posts:
41,454
Location:
Prague, Czechia

Re: Verifying the Host Certificate in Script with CA certificate

Makc666 wrote:

after all investigations I expect that WinSCP does NOT use Windows Certificate Storage.
WinSCP DOES USE Windows Certificate store.

If the source code from the above link is actual then is it really that cacert.pem file is the only place where WinSCP is looking for trusted CA certificates?
No. Use of the cacert.pem was deliberately never documented. It's not recommended for use.

If all points above are done correctly you don't have to use -hostkey="<fingerprint>" or -certificate="<fingerprint>" (I didn't test -hostkey but I expect it to work) for FTPS/SFTP.
SFTP/SSH does not use certificates. So all this is irrelevant for SFTP/SSH.

Reply with quote

Makc666
Joined:
Posts:
50
Location:
MSK-RU

Re: Verifying the Host Certificate in Script with CA certificate

martin wrote:

WinSCP DOES USE Windows Certificate store.
@martin,
I did my best to make WinSCP does use Windows Certificate store.
I added my Root-CA and SUB-CA certificates to Windows Certificate store but WinSCP doesn't see them.

In which part of the Store then I have to put them?
Where WinSCP is looking for those certificates then – in Global storage, Users storage, or may be in some other?

martin wrote:

No. Use of the cacert.pem was deliberately never documented. It's not recommended for use.
For now it is the only way to use self-signed certificates.
As about documented or not – I can't comment but from my point of view to hide the fact that there is such future even not recommended is not the best case. But it is just my point.

martin wrote:

SFTP/SSH does not use certificates. So all this is irrelevant for SFTP/SSH.
I edited this one in my original post. Thanks!

Reply with quote

martin
Site Admin
martin avatar
Joined:
Posts:
41,454
Location:
Prague, Czechia

Re: Verifying the Host Certificate in Script with CA certificate

Makc666 wrote:

In which part of the Store then I have to put them?
Where WinSCP is looking for those certificates then – in Global storage, Users storage, or may be in some other?
There's the CertVerifyCertificateChainPolicy(CERT_CHAIN_POLICY_SSL, ...) API that WinSCP uses:
https://learn.microsoft.com/en-us/windows/win32/api/wincrypt/nf-wincrypt-certverifycertificatechainpolicy

So WinSCP does not look anywhere specifically. It's all up to the above API.

Reply with quote

Makc666
Joined:
Posts:
50
Location:
MSK-RU

Re: Verifying the Host Certificate in Script with CA certificate

@martin: This API killed my mind today.

Yes, I can see WinSCP uses somehow the Windows Certificates Store.

If you don't have you third-party Root-CA and Sub-CA in Windows store you will get this error in WinSCP logs:
. 2017-02-06 15:38:00.824 Certificate failed to verify against Windows certificate store: Error: 800B0109, Chain index: 0, Element index: -1
When you add your Root-CA and Sub-CA in Windows store you will get this error in WinSCP logs:
. 2017-02-06 17:02:40.025 Certificate failed to verify against Windows certificate store: Error: 80092013, Chain index: 0, Element index: 0
For now I am not able to solve this problem.
The only thing I can tell that using
certutil -setreg ca\CRLFlags +CRLF_REVCHECK_IGNORE_OFFLINE
is not our case as it is only for your own CA you have at your server.

But for 3-rd party Root-CA and Sub-CA certificates this is a problem.

I understand that WinSCP sees the certificates but WinSCP can't do something.
And this something is not clearly for me.

As I have told that when I use cacert.pem it works perfectly.

I need help?! :)

Reply with quote

Advertisement

Makc666
Joined:
Posts:
50
Location:
MSK-RU

@martin, I found out the core of the problem.

Really it was simple one.

You have certificate which was issued by custom CA (Certificate Authority).
In this case as expected your Windows system doesn't have:
- Root certificate in "Trusted Root Certification Authorities" (I call it CA-ROOT);
- Intermediate certificate in "Intermediate Certification Authorities" (I call it CA-SUB) (SUB is from subordinate).

1. Error:
@sessionlog@
. 2017-02-06 15:38:00.824 Certificate failed to verify against Windows certificate store: Error: 800B0109, Chain index: 0, Element index: -1
Add into Windows Storage "certlm - [Certificates - Local computer]" (see screenshot) certificates of your custom CA:
- Root certificate in "Trusted Root Certification Authorities";
- Intermediate certificate in "Intermediate Certification Authorities".
(Start - Run/Search - certlm.msc)

2. Error:
@sessionlog@
. 2017-02-06 17:02:40.025 Certificate failed to verify against Windows certificate store: Error: 80092013, Chain index: 0, Element index: 0
Check:
Event Viewer (Local) - Windows Logs - System
for the Errors:
The IP Helper service depends on the WinHTTP Web Proxy Auto-Discovery Service service which failed to start because of the following error:
The service cannot be started, either because it is disabled or because it has no enabled devices associated with it.
The Network Connectivity Assistant service depends on the IP Helper service which failed to start because of the following error:
The dependency service or group failed to start.
If there is one, turn on (set to Automatic) and then start the service:

Service name: WinHttpAutoProxySvc
Display name: WinHTTP Web Proxy Auto-Discovery Service
Description: WinHTTP implements the client HTTP stack and provides developers with a Win32 API and COM Automation component for sending HTTP requests and receiving responses. In addition, WinHTTP provides support for auto-discovering a proxy configuration via its implementation of the Web Proxy Auto-Discovery (WPAD) protocol.

After that, start the service:

Service name: iphlpsvc
Display name: IP Helper
Description: Provides tunnel connectivity using IPv6 transition technologies (6to4, ISATAP, Port Proxy, and Teredo), and IP-HTTPS. If this service is stopped, the computer will not have the enhanced connectivity benefits that these technologies offer.

You in my case it start working!

You will get this line in WinSCP logs for success:
. 2017-02-13 15:47:41.487 Certificate verified against Windows certificate store
Last edited by Makc666 on 2017-02-14 13:19; edited 4 times in total

t=24406_03.png

t=24406_02.png

t=24406_01.png

Reply with quote

Advertisement

You can post new topics in this forum