Skip to content

Moved "no-shared" so that also windows statically link to the libraries #83

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Jan 7, 2021
Merged

Moved "no-shared" so that also windows statically link to the libraries #83

merged 4 commits into from
Jan 7, 2021

Conversation

molleafauss
Copy link
Contributor

Moved the argument and added it close to the "no-dso", as it's seems it's a "related" configuration switch.

@alexcrichton
Copy link
Owner

Thanks! Since I'm not entirely sure whether we have CI for it or not, have you confirmed this works for you both with and without crt-static?

@molleafauss
Copy link
Contributor Author

I'll double check.

@sfackler
Copy link
Collaborator

sfackler commented Jan 4, 2021

It looks like we do have CI coverage for that I think: https://github.com/alexcrichton/openssl-src-rs/blob/master/.github/workflows/main.yml#L99-L107

@molleafauss
Copy link
Contributor Author

Also, @alexcrichton should the crt-static workflow here be phased out given it was used only to check the two different linking version (wondering how it was working though...)?
https://github.com/molleafauss/openssl-src-rs/blob/master/.github/workflows/main.yml#L90

@sfackler
Copy link
Collaborator

sfackler commented Jan 4, 2021

I am a bit confused as to what changed from when this was written to now though: 3bce2e8. Presumably that was made for a good reason!

@alexcrichton
Copy link
Owner

Hm so C/C++ code needs to be compiled with a different flag depending on the crt-static value, and typically that happens through the cc crate but reviewing the code here it looks like cc may not be used and the no-shared/etc flags are what's being used. In that case @molleafauss when testing this I'd expect it to fail?

Looks like CI is at least busted in one way because there's failures which aren't failing CI. I'll look into that.

@molleafauss
Copy link
Contributor Author

Haven't looked into what tests are run in this crate, and how the built openssl lib is used.
I would indeed expect that the build missing the no-shared would fail, as it would not be able to load the openssl dlls, unless those libraries were available system-wide (i.e. via %PATH%) in the docker image used for testing?
But if the tests are only checking that the openssl sources are built properly, both shared and dll build for windows are working, the issue is with the upstream dependencies (openssl->openssl-src->this crate) and what they end up linking ultimately.
When the dll were built, then rustc was linking only the stubs to your binary, and then the binary would fail running, as the .dll were nowhere to be found.

@molleafauss
Copy link
Contributor Author

ouch - think I see the issue: this is in the windows build; both x86_64-msvc-static and x86_64-msvc. But the build passes... 🤦‍♂️

immagine

@molleafauss
Copy link
Contributor Author

And as I was suspecting this is the build environment: the OpenSSL 1.1.1 is included...

@alexcrichton
Copy link
Owner

Hm ok so testing this locally, I'm surprised that this ever worked actually. The problem the current static builds (pre-this-PR) is that a new dependency (user32.dll) was introduced and we're not linking it. That's easily fixed, however. The pre-this-PR non-static builds are actually doing entirely the wrong thing. They're attempting to statically link the import libraries, which is causing the duplicate symbol error. This crate was never intended to build shared libraries, so the fact that it's building shared libraries when +crt-static is missing is entirely a mistake.

This PR definitely fixes that issue and can be fixed in CI by adding an extra-linked library (user32.dll), but the problem still remains that the C code is compiled with /MT instead of /MD with -crt-static. I don't know how to fix that with OpenSSL's build system at this time, though. Somehow that needs to be fixed as well to fully fix this

@sfackler
Copy link
Collaborator

sfackler commented Jan 5, 2021

There is this thing that README.Windows mentions briefly you're supposed to compile into your application: https://github.com/openssl/openssl/blob/master/ms/applink.c. Maybe it does that to avoid dealing with static vs dynamic linkage to the crt?

@alexcrichton
Copy link
Owner

Ok so there's actually a whole mess of problems wrong with this crate and its CI right now:

  • CI doesn't actually ever enable crt-static testing, it's incorrect GitHub Actions configuration.
  • When crt-static is disabled, OpenSSL dynamic libraries are built but they're linked statically according to the info we provide rustc, this generates linking errors because you're not supposed to do that.
  • When compiled as a static library we're missing the -luser32 link argument, meaning that linking always fails if you actually enable crt-static.
  • CI was accidentally ignoring some failures on Windows due to using the wrong shell.
  • The directory with the dlls for OpenSSL wasn't actually added to Cargo's search path on MSVC.

I think those are all fixed with #84, however. Want to try rebasing over that and see what happens?

It may still be the case that +crt-static appears to work, but I don't think it will in general because there still isn't any supporting code to figure out how to compile libssl.lib with /MD for linking to libcmt.lib

@molleafauss
Copy link
Contributor Author

Did rebase the work, all tests are passing. I've removed the crt-static flag, given it seems pointless, basically making the configurations msvc and msvc-static identical.
I see that the tests are run.
Let me know your thoughts.

@alexcrichton
Copy link
Owner

Ok I did some testing locally and happened to stumble upon https://stackoverflow.com/questions/3469841/mixing-code-compiled-with-mt-and-md. Indeed OpenSSL is compiled with /Zl, and testing locally I can mix a /MT OpenSSL with a /MD object file and it appears to work. Given that OpenSSL seems like it may be passing that option intentionall then this looks good to me!

Sorry for the runaround with weird CI and such...

In any case, want to drop the shared flag now since it's always false? Other than that seems good to merge to me!

@molleafauss
Copy link
Contributor Author

no probs, will give a look later tonight (you might have guessed I'm on GMT here)

@molleafauss
Copy link
Contributor Author

Also should we remove the MSVC CI configs for the crt-static, leaving only 2 and not 4 for windows?

@alexcrichton
Copy link
Owner

Nah since that's different enough at the rustc level I'd prefer to not drop them just yet.

@alexcrichton alexcrichton merged commit 19f8b13 into alexcrichton:master Jan 7, 2021
neuronull added a commit to vectordotdev/openssl-src-rs that referenced this pull request Jul 19, 2022
* only build necessary target (alexcrichton#43)

* Bump to 111.6.1+1.1.1d

* Update checkout actions reference

* Fixed apparent typo in the readme (alexcrichton#46)

* Work around upstream cargo issues

* Add handling for target riscv64gc-unknown-linux-gnu (alexcrichton#48)

* Disable asmjs for now

* Allow to specify custom path for Perl (alexcrichton#49)

* Allow to specify custom path for Perl

Env var OPENSSL_SRC_PERL can be set to specify
a path to the perl binary to use to call the openssl
perl scripts.

Resolves alexcrichton#45

* Fallback to PERL env var if OPENSSL_SRC_PERL is not set

* Add support for non-x86 archs on FreeBSD (alexcrichton#50)

Caused by:
  process didn't exit successfully: `.../rust-bootstrap/work-aarch64/rustc-1.40.0-src/build/x86_64-unknown-freebsd/stage1-tools/release/build/openssl-sys-4b2d35028bf73953/build-script-main` (exit code: 101)
--- stdout
cargo:rustc-cfg=const_fn

--- stderr
thread 'main' panicked at 'don't know how to configure OpenSSL for aarch64-unknown-freebsd', .../rust-bootstrap/work-aarch64/rustc-1.40.0-src/vendor/openssl-src/src/lib.rs:178:18

* Update CI installation of Rust on macos~

* support for illumos systems (alexcrichton#52)

Resolves alexcrichton#51

* More comment out of asmjs

* Use gmake by default on all DragonFlyBSD, FreeBSD or Solaris targets (alexcrichton#53)

* Bump to 1.1.1e (alexcrichton#54)

* Add support for illumos triple (alexcrichton#56)

* Bump to 1.1.1f (alexcrichton#58)

* Release 111.8.1+1.1.1f

* Bump to 1.1.1g

* Do not overwrite AR and RANLIB env vars if set (alexcrichton#62)

* Add engine support for linux-gnu (alexcrichton#63)

* add engine support for linux-gnu

Signed-off-by: Xintao <hunterlxt@live.com>

* address comment

Signed-off-by: Xintao <hunterlxt@live.com>

* add blacklist os

Signed-off-by: Xintao <hunterlxt@live.com>

* add blacklist os

Signed-off-by: Xintao <hunterlxt@live.com>

* To check CI build info

Signed-off-by: Xintao <hunterlxt@live.com>

* add comments

Signed-off-by: Xintao <hunterlxt@live.com>

* Bump to 111.10.0+1.1.1g

* Fix build on macOS with latest cc crate (alexcrichton#67)

cc v1.0.58 broke the macOS build by including the "-arch" flag in the
default set of compiler flags. Strip it out, like we do for iOS targets.

Closes alexcrichton#66.

* Bump to 111.10.1+1.1.1g

* add optional features for less used algorithms (alexcrichton#68)

This commit disables by default a few of the weaker cryptographical algorithms into
a "weak-crypto" feature as well as some of the less used algorithms into
their own specific features.
These algorithms are not directly exposed through the rust-openssl crate.
The compilation of these can be re-enabled by selecting the desired features.
This should slightly reduce build time and library size.

Signed-off-by: Petre Eftime <petre.eftime@gmail.com>

* Bump to 111.10.2+1.1.1g

* Bump to 1.1.1h (alexcrichton#73)

* Add FreeBSD powerpc64le support (alexcrichton#75)

FreeBSD PowerPC64LE is a new target.

This is needed to cross-build cargo.

* Add upstream patch to allow building on aarch64-apple-darwin (alexcrichton#74)

* Use fs module convenience methods for MUSL patch

* Add upstream patch to allow building on aarch64-apple-darwin

Closes alexcrichton#72

* i586 support (alexcrichton#76)

* Bump to 111.12.0+1.1.1h

* Add aarch64-apple-darwin to CI (alexcrichton#77)

* OpenBSD support (alexcrichton#78)

* Update OpenSSL to 1.1.1i

* Update CI

* Remove now no-longer-necessary patches

* Support targets `armv7-unknown-linux-gnueabi/musleabi` (alexcrichton#80)

Support targets `armv7-unknown-linux-gnueabi` & `armv7-unknown-linux-musleabi`

* Make it compile for wasm32-wasi (alexcrichton#81)

* Catch more failures on Windows in CI (alexcrichton#84)

* Catch more failures on Windows in CI

* Link missing library for OpenSSL on MSVC

Looks like some functions may pull in user32 functions, so that library
needs to be linked.

* Try to fix msvc +crt-static builds

* Try to see all failures

* More output

* Fix setting crt-static

* More CI tweaks

* Add the bin dir to cargo's search path on MSVC

That's where it contains the actual dlls needed at runtime

* Moved "no-shared" so that also windows statically link to the libraries (alexcrichton#83)

* Moved "no-shared" so that windows statically link to the libraries

* try reapplying changes

* removed shared as it's always false.

Co-authored-by: molleafauss <lmollea@yahoo.it>

* Bump to OpenSSL 1.1.1j (alexcrichton#85)

* add nasm support for msvc (alexcrichton#87)

* add nasm support for windows-msvc

This will automatically detect whether nasm.exe is installed and
try to enable the assembly language routines. These can also be
disabled by set the `OPENSSL_RUST_NO_NASM` environment variable to
a non-zero value.

* don't use '>> $GITHUB_ENV' to overwrite PATH

* remove the windows check in CI

* give user more control on the env var

* add env var in CI, less acceptable values for env var

* fix path format for using bash on windows

* 'OPENSSL_RUST_USE_NASM' env var only accept 0 or 1

* Bump to 1.1.1k

* Add FreeBSD powerpc support (alexcrichton#90)

This is needed to support cargo cross-build for 32-bit powerpc on FreeBSD.

* Upgrade to GitHub-native Dependabot (alexcrichton#92)

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>

* Support targets `armv5te-unknown-linux-gnueabi/musleabi` (alexcrichton#94)

* Support targets `popwerpc64(le)-unknown-linux-musl` (alexcrichton#95)

* Support targets `mips64(el)-unknown-linux-muslabi64` (alexcrichton#96)

* Support target `s390x-unknown-linux-musl` (alexcrichton#97)

* Bump to openssl 1.1.1l (alexcrichton#100)

* Bump to 1.1.1m

* test on 1.1.1 branch

* Fix aarch64-apple-darwin CI (alexcrichton#116)

(cherry picked from commit 466ffd2)

* Bump to 1.1.1n (alexcrichton#123)

* Update "old" windows image on CI (alexcrichton#126)

* Bump to 1.1.1o (alexcrichton#136)

* Bump to 1.1.1o

* Disable arm-linux-androideabi test

* Backport wycheproof exclude to 1.1.1 branch (alexcrichton#139)

* Backport wycheproof exclude to 1.1.1 branch

* Backport exclude CI checks

* Backport exclude CI checks

* Configure `--openssldir` to its default location (alexcrichton#141) (alexcrichton#142)

As pointed out in alexcrichton#140 otherwise certificates and configure is by
default looked up in the directory of the build machine itself which is
often a writable path. Instead switch the configuration option back to
the default recommended in openssl's `INSTALL.md`

Closes alexcrichton#140

* Bump to 111.20.0+1.1.1o (alexcrichton#143)

* Bump to 1.1.1p (alexcrichton#145)

* Bump to 1.1.1q (alexcrichton#147)

Co-authored-by: Jay <BusyJay@users.noreply.github.com>
Co-authored-by: Alex Crichton <alex@alexcrichton.com>
Co-authored-by: Alex Gaynor <alex.gaynor@gmail.com>
Co-authored-by: msizanoen1 <55322658+msizanoen1@users.noreply.github.com>
Co-authored-by: Franck Royer <royer.franck@gmail.com>
Co-authored-by: Tobias Kortkamp <t6@users.noreply.github.com>
Co-authored-by: Joshua M. Clulow <josh@sysmgr.org>
Co-authored-by: MikaelUrankar <49529234+MikaelUrankar@users.noreply.github.com>
Co-authored-by: Steven Fackler <sfackler@gmail.com>
Co-authored-by: Patrick Mooney <pmooney@pfmooney.com>
Co-authored-by: Steven Fackler <sfackler@palantir.com>
Co-authored-by: James McMurray <jamesmcm03@gmail.com>
Co-authored-by: Xintao <hunterlxt@live.com>
Co-authored-by: Nikhil Benesch <nikhil.benesch@gmail.com>
Co-authored-by: petreeftime <petre.eftime@gmail.com>
Co-authored-by: Brandon Bergren <git@bdragon.rtk0.net>
Co-authored-by: Jake Goulding <shepmaster@mac.com>
Co-authored-by: Will <will@mon.im>
Co-authored-by: Jake Goulding <jake.goulding@gmail.com>
Co-authored-by: kiron1 <kiron1@gmail.com>
Co-authored-by: IceCodeNew <32576256+IceCodeNew@users.noreply.github.com>
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
Co-authored-by: Alex Malgaroli <alex.malgaroli@gmail.com>
Co-authored-by: molleafauss <lmollea@yahoo.it>
Co-authored-by: Kane <KaneGreen@users.noreply.github.com>
Co-authored-by: Brandon Bergren <bdragon@FreeBSD.org>
Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
Co-authored-by: messense <messense@icloud.com>
Co-authored-by: Alexis Mousset <contact@amousset.me>
Co-authored-by: Eric Huss <eric@huss.org>
Co-authored-by: Alexis Mousset <alexis.mousset@rudder.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants