-
Notifications
You must be signed in to change notification settings - Fork 9
Adding Support for OpenSSL 3.x and Lazarus #22
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
…hi subdirs for pascal source and delphi build files respectively.
…to include updated unit files
|
Hi. How to correctly install this package with main Indy package from git? Delphi 12. When I try to build this package "as is" after installing Indy package from git (successfully), first errors that appear is missing RSOSSLEVPDigestExError and RSOSSLEVPDigestUpdateError from IdResourceStringsOpenSSL.pas. This is because IndyCore package already contains IdResourceStringsOpenSSL.pas without these constants. Next comes the MD5 and SHA errors, function declarations differ from previous ones. I assume same thing there, those functions were already declared earlier. What I'm actually having trouble to understand, is how to merge https://github.com/IndySockets/Indy and https://github.com/MWASoftware/IndyTLS-OpenSSL in order to test them. Regards. |
|
If you are trying to test out my proposed OpenSSL add on for Indy, then you must use the IndySockets Indy10.7 branch and not the master branch. Please check that you are on the correct branch.
|
|
Ouch, missed that one. Will do, tyvm! |
|
I think I got it with Indy-10.7 branch, but not sure about Linux.. When I add IdSSLOpenSSL in uses in my project, compiler tries to compile this: Meaning that Windows is defined, while chosen platform is Linux. What have I missed? |
|
Got this when trying to compile your test project I think there is a miss with package naming somewhere. Is it IndyOpenSSL290 or IndyTLSOpenSSL290? :-D |
|
Ah, you are obviously using Delphi. There is a naming mismatch between Lazarus/fpc and Delphi which I failed to spot. I predominately use the former, but I do try to also repeat the tests with Delphi. Looks like I missed this one. I will update and ensure consistent package names. In the meantime, IndyTLSOpenSSL290 is the correct package name for Delphi. |
|
Great, thanks! What about this snippet that is being compiled under Linux platform: Do you have any idea? Regards. |
That is surprising WINDOWS is defined in the IdCompilerDefines.inc file that is standard across Indy. My Linux testing is with Lazarus/fpc and I have not seen this problem. What is your Linux build environment? |
|
I use Ubuntu 24.04 x64 as SDK imported with PAServer 23.0 in Delphi 12. You are right, IdCompilerDefines.inc defines WINDOWS if compiler is not FPC. So basicaly need to add more defines related to chosen build platform. I'll play with this file, thank you! |
Yes, but only if |
|
Same with There seems to be a hardcoded rule in many places: When compiler is not FPC, then we are using Windows :-P Then comes the following code:
I hope we are getting one step closer to Linux with this discussion :-) |
When I create a new project in my Delphi setup, without using Indy or IndyOpenSSL, and setting target platform to Linux, I get this result: EDIT: So I have to include |
I think I've found the problem. On a clean install of Windows-11 and Delphi 12 (CE), I found a problem with the order in which the package searches for the OpenSSL libraries. Instead of picking up the expected versions, it found a libcrypt.dll in Windows\System32. This did not seem to be compatible. I have updated the search order in the updated version so that a version without a suffix is only picked up if there is no version with a suffix e.g. libcrypt-3-x64.dll, |
Hm, aren't |
|
You are correct. I missed the 'o' out in my comment. The correct name is used by the software. |
…any suffix, unless an empty suffix is included in the list of suffixes. For Linux, the default empty suffix is the first in the list (e.g. looks for libcrtypo.so first. For Windows, libcrypto.dll is the last in the list.
Well, coming from a Safety Critical background where long term software support is essential, I'm impressed by your commitment to supporting a 25 year old product. I can just about remember Delphi 5 (I probably have the installation CD somewhere), but I upgraded to Delphi 6 almost as soon as it came out in 2001. This is probably a good time to drop support for a compiler that does not support the {$if declared... and {$if defined,,,} syntax. Checking my proposed IndyTLS-OpenSSL package, I have made some use of {$if declared for type definitions. It's a very neat way of dealing with type definitions across multiple platforms and compilers. I could regress to just use of $IFDEF, but it would mean introducing all that clunky stuff in IdCompilerDefines.inc plus the maintenance headache implied. Peter will probably have an even bigger problem with TaurusTLS if you keep that restraint. He took a snapshot of my OpenSSL headers early on in their development, and that iteration made very extensive use of both {$if defined and {$if declared conditionals. I've checked and they are still there in his repo. In later versions of the headers, I introduced the "Just in time" approach to function loading and that resulted in simpler code that did not need or use {$if. He will need to reintegrate the current versions of the OpenSSL headers. A quick search of TaurusTLS.pas also revealed extensive use of this syntax. |
In more testing, I have found that with win64 in particular, it is very easy to pick up the libcrypto.ddl in Windows\system32 if you have not been careful to ensure that your version (e.g. libcrypto-3-x64.dll) is not properly installed. The result is the type of error @RoLex has been reporting. This is not great as the non-expert user will be given a misleading error report. I am think of taking a more limited default approach to which filenames are permissible i.e. restricting those acceptable to libcrypto-1-x64.dll and libcrypto-3-x64.dll, whilst allowing the programmer to extend the list if they want to. The problem has been that Windows Openssl dlls have had many different names over the years, starting with libeay32.dll. Maybe now is the time to be less permissive with the default dll names that are searched, and add a good readme on the issue. |
It's only because I know that there are still Indy users out there using such old versions. But I'm also not against cutting them off at some point. Have to move forward eventually.
I'm aware. It's already implemented for Indy 11, where the minimum versions are moved up to Delphi 2009 and FPC 3.0. Might be worth backporting some of the changes for 11 into 10.7. Not sure yet. I need to re-sync the Indy 11 code with the latest Indy 10 code.
AFAIK, he's not targetting old Delphi versions. |
Is it possible to supply library name/path to the OpenSSL reader class? This was possible with PR#299, and the idea was good. I'm thinking first of all about people who will use my application. They will probably run on very old OS that does not support installing packages with new version of OpenSSL library. Now I'm talking about Linux with default package installer. |
Yes, there is an interface that allows control over the library loading. There is an example of use in the test programs, but I probably should add a README on its use. The interface is in IdSSLOpenSSLAPI.pas and is: type
IOpenSSLDLL = interface(IOpenSSL)
['{1d6cd9e7-e656-4981-80d2-288b12a69306}']
procedure SetOpenSSLPath(const Value: string);
function GetSSLLibVersions: string;
procedure SetSSLLibVersions(AValue: string);
function GetLibCryptoHandle: TLibHandle;
function GetLibSSLHandle: TLibHandle;
function GetLibCryptoFilePath: string;
function GetLibSSLFilePath: string;
function GetFailedToLoadList: TStrings;
function Load: Boolean;
procedure Unload;
function IsLoaded: boolean;
property SSLLibVersions: string read GetSSLLibVersions write SetSSLLibVersions;
end;
function GetIOpenSSLDDL: IOpenSSLDLL;The interface allows you to specify the directory to search for the dll/so and the list of possible suffices for the library name. Reviewing the interface, it does not allow you to override the base name for the dll/so e,g, libcrypto, libssl. Perhaps that should be included. The default SSLpath is empty i.e. use the system default search path, and the default suffices list is (list in preference order): DefaultLibVersions = ':.3:.1.1:.1.0.2:.1.0.0:.0.9.9:.0.9.8:.0.9.7:.0.9.6'; //Linux
DefaultLibVersions = '-3-x64;-1-x64;'; //Windows 64-bit
DefaultLibVersions = '-3;-1;'; //Windows 32-bitNote the empty list elements at the start of the Linux list and end of the Windows lists. I am thinking of removing empty list element from the Windows lists but not the Linux lists. Generally, Linux has fewer issues here than Windows. Most distros use the standard libnames from the OpenSSL package and so you get e.g. libssl.so.3 and a symlink from libssl.so to libssl.so.3. If you want to use a non-standard build, its often easier to set the LD_LIBRARY_PATH environment variable than it is to use the IOpenSSLDLL interface - the choice is yours. |
I suppose this begs the question as to what your intentions are with the IndyTLS-OpenSSL package. Is it also constrained to support Delphi 5, or is it allows to use{$if ...? |
I've added READMe.OpenSSL. Most of the text already existed, so quickly done. Also extended the IOpenSSLDLL interface to allow for lib base name changes. |
My plan was for:
But before that could happen, JP created TaurusTLS, so IndyTLS-OpenSSL might not even get a 2.0 now (1.0 for sure, that's a done deal for Indy 10.7). I'm not sure yet, I'm still open to the possibility of a 2.0 if it makes sense. Or maybe merge TaurusTLS and IndTLS-OpenSSL 2.0 together, who knows. |
|
But do you expect TaurusTLS or IndyTLS v2 to still support Delphi 5?
|
|
I would also argue that there needs to be more haste in terms of Indy supporting a current version of OpenSSL. All my use of Indy is for REST type services over https, and I suspect that the majority of users want that as well. Not being able to support OpenSSL 3 is a major deficiency in Indy 10.6. You need to get Indy 10.7 with OpenSSL 3 support out of the door ASAP. |
|
I also have no idea what Indy project developers are waiting for. If the work on new OpenSSL began long time ago, there would not appear concurrents like Taurus or eSeGeCe. They did appear because people needed TLS 1.3 as the rest of the world is moving forward. At the same time, I also understand that Remy does not have time needed to perform this large work on Indy - a free project, he said that many times before. But right now there is barely need of doing anything, the code has already been written! Also Indy has the opportunity that noone else have - it is included with RAD studio by default. My suggestion on milestone:
But this needs to be done fast, so people get time to test it. By that time we are hoping to see Indy 11 in RAD studio 13 updates. |
Thank you for what you have said as well as helping to tease out some remaining multiplatform issues from my fork of the IndyTLS-OpenSSL package. I can very much appreciate the problem that Remy has as the single maintainer of Indy. For the last 15 years, I have been maintaining the Lazarus fork of IBX and when you sign up to this, you sign up to dealing with user support, bug and fixes and, in my case, keeping up-to-date with the many changes that have taken place in the Firebird RDMBMS since it split from InterBase. It's a serious commitment. On the other hand, in my professional (i.e. paid for) work, I have several times had to deploy the aphorism "don't let the perfect be the enemy of the good" - and at times like this. You do need to test software before release, and it's why I put a lot of emphasis on a properly constructed test plan that includes the capability for regression testing. But, until a product starts being tested in the field, you cannot get full confidence in its operation. This is why you have alpha and beta pre-releases. IHMO, Indy with OpenSSL 3.x support is more than ready for alpha or even beta release and has been so for some time. It's now more than a year since Remy asked Peter Mugaas and myself to help add OpenSSL 3.x support. A strawman package was produced and then Peter and myself went down different paths using the same initial package. Peter produced an add-on to Indy 10 which required changing the unit and class names (TaurusTLS). I developed a fork of Indy 10 with OpenSSL 3.x support (Indy.ProposedUpdate) and kept this is up-to-date with Indy 10 commits. I also copied the same OpenSSL units to a fork of Remy's proposed IndyTLS-OpenSSL package for use with the putative Indy 10.7 (and which you have been testing). At the very least, Indy users had a choice of upgrade path. In terms of the code, mine and the TaurusTLS software has diverged since then. I would point to the work I have done since the initial strawman on improving the OpenSSL library load strategies with the faster "Just in Time" approach. I have also done some code clean up in the meantime in respect of the Indy OpenSSL code, and put emphasis on ensuring that both Delphi and Lazarus are fully supported by the same codebase - plus many bug fixes For Indy users, there are two alternatives available at present: use Indy 10 with a separate add-on and your change class and unit names, or fully replace the distributed Indy 10 with my Indy.Proposed update fork and avoid name changes. That's the user's choice. However, it would by much better if there was an "offical" path that was properly distributed with the next update of Delphi and via the Lazarus Online Package Manager (OPM). I don't see why Indy 10.7 can't be made available now (label it an alpha if you wish - just get it out there). As to whether the OpenSSL support path is via TaurusTLS or IndyTLS-OpenSSL - well naturally I favour the latter. But if TaurusTLS was preferred then I would be happy to offer to fold in the work I have done on improvements to the OpenSSL headers, library loading and code clean up - along with comparing bug fixes, into TaurusTLS. The important thing is to get something out there. Finally, I have one further vested interest to declare. I have an oauth2client package on github that is accumulating stars and depends on Indy. I would really like to make this available via the Lazarus OPM. Indy 10 is in the OPM and there is no point in distributing this package if it can then only work with obsolete OpenSSL libraries and provides at most TLS 1.2 (now often deprecated). What I want is to get Indy 10.7 and IndyTLS-OpenSSL distributed via the OPM. |
|
Btw, Win64 now works with Delphi 12! =) I guess it's time to get rid of last small warnings and hints from Delphi compiler: |
Certainly not.
I'm aware of that. This has been an ongoing problem for several years now.
I'm the only project developer left. Its not that I'm waiting, its that I haven't had free time to work on it.
I agree.
It's not just writing the code, but also reviewing and testing the MASSIVE amount of changes that this update involves. And that I have not been able to focus on. Hence why I'm having Indy 10.7 break out OpenSSL completely from the main library, so its isolated and easier for community members to work on going forward.
That is not as big an opportunity as you think it is.
Embarcadero usually bundles Indy updates only in major IDE releases. Non-interface-breaking bug fixes might get into minor IDE updates, but not big version changes. Even if I were to publish Indy 10.7 soon, or even Indy 11 (which BTW is another large Indy update), Embarcadero would not be able to bundle it until the next major IDE release (14+). So don't expect to see 10.7 or 11 appear in RAD Studio 13 updates.
It has been in various forms, but getting a final final final release ready has been a challenge for me.
There have been a couple of other 3rd party implementations, too.
That's what I've been trying to get to. |
…d to use MD5 functions
…ndows environments
|
c34ca1e#diff-b9aefaa16cc17306ea293b47b3ab51bec71111dc609f055bcefdd0f7680ce3a2R9 @MWASoftware Here you set Win64 by default, which makes it impossible to build and install your package due to missing Indy packages. This is because Indy packages are built and installed for Win32 by default. |
|
Thanks. Will update. I'm checking through the install and test procedures on various platforms and will feed this in.
|
|
Its clear to me from @rlebeau's post that 10.7 is still some way off. I have also concluded that there needs to be a support path for Delphi users which will have 10.6 for the foreseeable future. @JPeterMugaas was probably correct in splitting off to create TaurusTLS. On the other hand, I don't want to waste all the work that has gone into create my fork of IndyTLS-OpenSSL. I have thus created a new version with the names changed to ensure that it can be used alongside Indy 10.6. You can find this at https://github.com/MWASoftware/IndySecOpenSSL I have so far tested it with Delphi 12 and 10, both with the the supplied Indy and replaced with Indy 10.7. I have also tested it with Lazarus on both Linux and Windows. Mindful of @RoLex's comments on Win32. I have gone through the package project files and the whole thing seems much slicker. THere are also GUI examples. I have also taken the opportunity to write a comprehensive User Guide. The User Guide is 68 pages long as I have added my own text (as appendicies) analysing TLS 1.3 and DTLS, and a tutoral on Security Concepts. The main body of the User Guide is intended to be an indepth guide to installation and use, please a reference section. I have also ended support for my fork of IndyTLS-OpenSSL and Indy.ProposedUpdate with all future work focused in the new package. |

This pull request provides a full update ot the package. It is based on the OpenSSL headers generated by the code generator provided by https://github.com/MWASoftware/PascalAPI4OpenSSL
This version supports OpenSSL 3.x, 1.1.1 and 1.0.2 for dynamic library loading. It also supports both Lazarus/fpc and Delphi. Package build files are provided for each toolchain.
Three link models are supported.
For dynamic library load, a "Just in Time" approach is used where each API call is initialised to a local proc "loader" function.
The intent is that on the first call a given API function, the actual entry point in the OpenSSL function
is loaded and the API call is set to the loaded entry point. The API function is now called on the user's
behalf. If the call fails to load then it is replaced by a compatibility function (if one exists). If none exists
then an exception is raised. If an API function is allowed to be nil (as set in the template), then the function
is loaded at library load time.
See the README file for further information.