Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Doesn't build on Windows #231

Open
LRN opened this issue Apr 24, 2019 · 24 comments
Open

Doesn't build on Windows #231

LRN opened this issue Apr 24, 2019 · 24 comments

Comments

@LRN
Copy link

LRN commented Apr 24, 2019

There are multiple problems with building appstream on Windows with MinGW-w64:

  • Includes <sys/utsname.h> (missing on Windows) unconditionally.
    • Should use ccompiler.has_header() to detect it at configure-time, then conf.set('HAVE_SYS_UTSNAME', result-of-the-check)
  • Uses <fnmatch.h> (missing on Windows) for `fnmatch().
    • Should use <glib/gpattern.h> and g_pattern_match_simple(), or something along these lines.
  • as_get_current_arch() uses utsname (see above) and needs a Windows-specific implementation.
    • A simple GetSystemInfo() call should do the trick (though be mindful of PROCESSOR_ARCHITECTURE_* constants, some of them are not in MinGW-w64 headers yet).
  • directory_is_empty() uses opendir() & friends. Doesn't work with utf-8 filenames, and only works on Windows because MinGW has an implementation (so no MSVC compatibility).
    • Replace with g_dir_open() & friends.
  • as_utils_is_root() uses unportable means of checking for administrative privileges (checks UID).
    • There's some code (see Solution #3) that you can re-use, although i would suggest getting rid of __try() & friends, as these don't work with MinGW.
  • as-distro-extras.c uses symlink() (absent on Windows)
    • It's easy to approximate with CreateSymbolicLinkW() (Vista-or-later required - add_global_arguments ('-D_WIN32_WINNT=0x600', language : 'c')), with a bit of extra code for utf-8->utf-16 conversion (glib has that), though be sure to add SYMBOLIC_LINK_FLAG_ALLOW_UNPRIVILEGED_CREATE flag. Note that you also need a special flag to make a directory symlink, so this function works either on files only or on directories only, depending on the flags. Appstream seems to use symlink() only for files, and when it won't, the caller would surely know what is being linked, so the appropriate context information can be passed down.
  • ascli-actions-mdata.c uses uint instead of gint (causes a warning)
  • appstream-cli.c uses isatty(fileno (stdout)) - very problematic on Windows.
    • Use g_log_writer_supports_color (fileno (stdout)) instead.
  • Meson builddefs add unconditional 'gio-unix-2.0' dependency
    • Make it conditional on host_machine.system() != 'windows'. No need for gio-win32-2.0 or anything in the else branch, AFAICS.
  • Meson builddefs have bizarre include_directories(['/usr/include']).
    • Just remove that.
  • as_distro_details_get_str() crashes because it can call g_key_file_get_string() with NULL key and id.
    • Check for priv->keyf != NULL && priv->id != NULL, return NULL if the check fails. Same goes for other places where priv->id or priv->keyf are used - add NULL checks.
@ximion
Copy link
Owner

ximion commented Apr 24, 2019

AppStream/libappstream has not at all been designed to work on Windows, and I have no means to test/debug this properly (especially since I have no reason to attempt a Windows port).
However, patches to make this work would be welcome!

@LRN
Copy link
Author

LRN commented Apr 25, 2019

I can't fork on github and file a MR, since that would require me to enable non-free JS on github. If you're OK with patches-disguised-as-text-files (because github really wants me to fork and forbids attaching .patch files), i can attach these.

P.S. "Not been designed" is not entirely true. Appstream is distro-agnostic by definition, and that can be stretched to cover Windows as well.

P.P.S. That said, i have absolutely no clue who or why would want to use libappstream on Windows. But it can be built, so it should be!

@hughsie
Copy link
Collaborator

hughsie commented Apr 25, 2019

If it helps, I think the gimp people are building appstream-glib on Windows.

since that would require me to enable non-free JS on github

You know Windows and the WIN32 API is non-free, right?

But it can be built, so it should be

If you're willing to write the code... you can't expect @ximion to just do all that work.

@LRN
Copy link
Author

LRN commented Apr 25, 2019

If you're willing to write the code...

if you're OK with patches-disguised-as-text-files [snip], i can attach these.

@ximion
Copy link
Owner

ximion commented Apr 25, 2019

You could also clone the repository to somewhere else and have me pull from there (I wouldn't mind patches as textfiles either though, as long as I can comment on them).
AppStream is distribution-agnostic, but it is designed for Linux distributions primarily and may be used by *BSD, if they want it. That's what it is tested on and what it is developed for.
What concerns me a bit is that you would port the library and tools "just because" - I don't consider this to be a long-term viable approach, because the code will inevitably become unbuildable on Windows again at some point if nobody actually uses in on Windows, and more importantly, if nobody would use it on Windows, we would have zero testing on that platform. If it compiles, it doesn't necessarily mean that everything also works as expected.

That said, if there is a benefit for someone and the patches to the code aren't too invasive, I don't mind merging such changes. Merging code that nobody uses though, is something I am wary about.

ximion added a commit that referenced this issue Jun 16, 2019
@Conan-Kudo
Copy link
Contributor

I can't fork on github and file a MR, since that would require me to enable non-free JS on github. If you're OK with patches-disguised-as-text-files (because github really wants me to fork and forbids attaching .patch files), i can attach these.

@LRN You know you can do forking, pushing, and making pull requests entirely through GitHub's API? There's even a program for this called hub.

@taozuhong
Copy link
Contributor

taozuhong commented Jun 6, 2023

execv ==> g_spawn_sync
symlink ==> g_file_make_symbolic_link(g_file_new_for_path())

Now just sys/utsname.h will break building,the output message as follow:

$ ninja
[2/18] Compiling C object src/libappstream-5.dll.p/as-system-info.c.obj
FAILED: src/libappstream-5.dll.p/as-system-info.c.obj
"ccache" "cc" "-Isrc/libappstream-5.dll.p" "-Isrc/" "-I../src" "-I." "-I.." "-Isrc" "-IC:/msys/ucrt6
4/include/glib-2.0" "-IC:/msys/ucrt64/lib/glib-2.0/include" "-IC:/msys/ucrt64/include/gio-win32-2.0"
 "-IC:/msys/ucrt64/include/libxmlb-2" "-IC:/msys/ucrt64/include/libxml2" "-fdiagnostics-color=always
" "-D_FILE_OFFSET_BITS=64" "-Wall" "-Winvalid-pch" "-std=c11" "-O0" "-g" "-Werror=shadow" "-Werror=e
mpty-body" "-Werror=strict-prototypes" "-Werror=missing-prototypes" "-Werror=implicit-function-decla
ration" "-Werror=pointer-arith" "-Werror=missing-declarations" "-Werror=return-type" "-Werror=int-co
nversion" "-Werror=incompatible-pointer-types" "-Werror=misleading-indentation" "-Werror=missing-inc
lude-dirs" "-Werror=declaration-after-statement" "-Wno-missing-field-initializers" "-Wno-error=missi
ng-field-initializers" "-Wno-unused-parameter" "-Wno-error=unused-parameter" "-D_POSIX_C_SOURCE=2008
09L" "-DAS_COMPILATION" -MD -MQ src/libappstream-5.dll.p/as-system-info.c.obj -MF "src/libappstream-
5.dll.p/as-system-info.c.obj.d" -o src/libappstream-5.dll.p/as-system-info.c.obj "-c" ../src/as-syst
em-info.c
../src/as-system-info.c:42:10: fatal error: sys/utsname.h: No such file or directory
   42 | #include "sys/utsname.h"
      |          ^~~~~~~~~~~~~~~
compilation terminated.
[3/18] Compiling C object src/libappstream-5.dll.p/as-utils.c.obj
FAILED: src/libappstream-5.dll.p/as-utils.c.obj
"ccache" "cc" "-Isrc/libappstream-5.dll.p" "-Isrc/" "-I../src" "-I." "-I.." "-Isrc" "-IC:/msys/ucrt6
4/include/glib-2.0" "-IC:/msys/ucrt64/lib/glib-2.0/include" "-IC:/msys/ucrt64/include/gio-win32-2.0"
 "-IC:/msys/ucrt64/include/libxmlb-2" "-IC:/msys/ucrt64/include/libxml2" "-fdiagnostics-color=always
" "-D_FILE_OFFSET_BITS=64" "-Wall" "-Winvalid-pch" "-std=c11" "-O0" "-g" "-Werror=shadow" "-Werror=e
mpty-body" "-Werror=strict-prototypes" "-Werror=missing-prototypes" "-Werror=implicit-function-decla
ration" "-Werror=pointer-arith" "-Werror=missing-declarations" "-Werror=return-type" "-Werror=int-co
nversion" "-Werror=incompatible-pointer-types" "-Werror=misleading-indentation" "-Werror=missing-inc
lude-dirs" "-Werror=declaration-after-statement" "-Wno-missing-field-initializers" "-Wno-error=missi
ng-field-initializers" "-Wno-unused-parameter" "-Wno-error=unused-parameter" "-D_POSIX_C_SOURCE=2008
09L" "-DAS_COMPILATION" -MD -MQ src/libappstream-5.dll.p/as-utils.c.obj -MF "src/libappstream-5.dll.
p/as-utils.c.obj.d" -o src/libappstream-5.dll.p/as-utils.c.obj "-c" ../src/as-utils.c
../src/as-utils.c:37:10: fatal error: sys/utsname.h: No such file or directory
   37 | #include <sys/utsname.h>
      |          ^~~~~~~~~~~~~~~
compilation terminated.
[4/18] Compiling C object src/libappstream-5.dll.p/as-curl.c.obj
FAILED: src/libappstream-5.dll.p/as-curl.c.obj
"ccache" "cc" "-Isrc/libappstream-5.dll.p" "-Isrc/" "-I../src" "-I." "-I.." "-Isrc" "-IC:/msys/ucrt6
4/include/glib-2.0" "-IC:/msys/ucrt64/lib/glib-2.0/include" "-IC:/msys/ucrt64/include/gio-win32-2.0"
 "-IC:/msys/ucrt64/include/libxmlb-2" "-IC:/msys/ucrt64/include/libxml2" "-fdiagnostics-color=always
" "-D_FILE_OFFSET_BITS=64" "-Wall" "-Winvalid-pch" "-std=c11" "-O0" "-g" "-Werror=shadow" "-Werror=e
mpty-body" "-Werror=strict-prototypes" "-Werror=missing-prototypes" "-Werror=implicit-function-decla
ration" "-Werror=pointer-arith" "-Werror=missing-declarations" "-Werror=return-type" "-Werror=int-co
nversion" "-Werror=incompatible-pointer-types" "-Werror=misleading-indentation" "-Werror=missing-inc
lude-dirs" "-Werror=declaration-after-statement" "-Wno-missing-field-initializers" "-Wno-error=missi
ng-field-initializers" "-Wno-unused-parameter" "-Wno-error=unused-parameter" "-D_POSIX_C_SOURCE=2008
09L" "-DAS_COMPILATION" -MD -MQ src/libappstream-5.dll.p/as-curl.c.obj -MF "src/libappstream-5.dll.p
/as-curl.c.obj.d" -o src/libappstream-5.dll.p/as-curl.c.obj "-c" ../src/as-curl.c
In file included from ../src/as-curl.c:27:
./config.h:16:17: error: expected identifier or '(' before string constant
   16 | #define DATADIR "C:/msys/ucrt64/share"
      |                 ^~~~~~~~~~~~~~~~~~~~~~
ninja: build stopped: subcommand failed.

@ximion
Copy link
Owner

ximion commented Jun 6, 2023

execv ==> g_spawn_sync

Those are not the same at all though... (unless paired with a fork() and no other options)
Doesn't MSYS2 provide execv?

Now just sys/utsname.h will break building

That will likely need a Windows-specific implementation to get details about the Windows OS...

@alex-tee
Copy link

alex-tee commented Sep 8, 2023

does this fix this issue? #510

@taozuhong
Copy link
Contributor

taozuhong commented Sep 8, 2023

Can't pass the compilation with default options on Windows(MSYS2).

Disable the stemming and systemed options could pass the compilation.

@alex-tee
Copy link

alex-tee commented Sep 8, 2023

one of the problems is hardcodig /usr/include in https://github.com/ximion/appstream/blob/main/meson.build#L167

passing -Dstemming=false works around it for now. another problem I had was docs failing to build (and they also pull in an extra dependency xsltproc). so i made a fork with this commit to skip that for now: https://gitlab.zrythm.org/zrythm/appstream/-/commit/ee2c3545ec6ed708121a7c9ea28252afbc0da8e1

rebuilding now... let's see

@ximion
Copy link
Owner

ximion commented Sep 8, 2023

Are manual pages a useful thing to have on Windows?

@alex-tee
Copy link

alex-tee commented Sep 8, 2023

not for me, i just want to build libadwaita, but i'm sure MSYS2 would like to have manpages like every other software it packages

@alex-tee
Copy link

alex-tee commented Sep 9, 2023

MSYS build succeeded for me after skipping docs and disabling stemming as mentioned above

PKGBUILD file posted here: msys2/MINGW-packages#17448 (comment)

@ximion
Copy link
Owner

ximion commented Sep 9, 2023

not for me, i just want to build libadwaita, but i'm sure MSYS2 would like to have manpages like every other software it packages

In that case I would really like to keep them default-enabled.
For the stemming issue we could probably add some Windows-specific logic. Or maybe make Snowball finally use a proper pkg-config file.

@alex-tee
Copy link

alex-tee commented Sep 9, 2023

@ximion re stemming: libstemmer and snowball are already packaged in MSYS2 so they are usable: https://packages.msys2.org/package/mingw-w64-ucrt-x86_64-libstemmer?repo=ucrt64

the only problem is the meson config using a hardcoded include dir that doesn't exist on MSYS2 systems (iirc it's /ucrt64/include or something like that: https://github.com/ximion/appstream/blob/main/meson.build#L167

using the standard include dir from meson should fix it: get_option ('prefix') / get_option ('includedir') instead of '/usr/include'

@ximion
Copy link
Owner

ximion commented Sep 9, 2023

get_option ('prefix') / get_option ('includedir')

This will fail horribly on Linux systems where you build for a prefix that isn't /usr while the stemming library is in /usr, which is a very common case.

@alex-tee
Copy link

alex-tee commented Sep 9, 2023

how about this then? although i believe the best course of action is to add a meson option to explicitly specify where to look for it, defaulting to get_option ('prefix') / get_option ('includedir')

diff --git a/meson.build b/meson.build
index 0b5face2..2b738f41 100644
--- a/meson.build
+++ b/meson.build
@@ -141,6 +141,14 @@ endif
 
 add_project_arguments('-DAS_COMPILATION', language: 'c')
 
+#
+# Modules
+#
+fs   = import('fs')
+glib = import('gnome')
+i18n = import('i18n')
+pkgc = import('pkgconfig')
+
 #
 # Dependencies
 #
@@ -164,27 +172,24 @@ endif
 stemmer_inc_dirs = include_directories()
 if get_option('stemming')
     stemmer_lib = cc.find_library('stemmer', required: true)
-    stemmer_inc_dirs = include_directories(['/usr/include'])
-    if not cc.has_header('libstemmer.h')
-        if cc.has_header('libstemmer/libstemmer.h')
-            stemmer_inc_dirs = include_directories('/usr/include/libstemmer')
-        else
-            error('Unable to find Snowball header "libstemmer.h". Please ensure libstemmer/Snowball is installed properly in order to continue.')
-        endif
+    stemmer_check_inc_dirs = [
+      '/usr/include', '/usr/include/libstemmer', get_option('prefix') / get_option('includedir')]
+    found = false
+    foreach stemmer_check_inc_dir : stemmer_check_inc_dirs
+      if fs.is_dir(stemmer_check_inc_dir) and cc.has_header('libstemmer.h', include_directories: include_directories(stemmer_check_inc_dir))
+        stemmer_inc_dirs = include_directories(stemmer_check_inc_dir)
+        message('Found "libstemmer.h" in ' + stemmer_check_inc_dir)
+        found = true
+      endif
+    endforeach
+    if not found
+      error('Unable to find Snowball header "libstemmer.h". Please ensure libstemmer/Snowball is installed properly in order to continue.')
     endif
 endif
 
 # use gperf for faster string -> enum matching
 gperf = find_program('gperf')
 
-#
-# Modules
-#
-fs   = import('fs')
-glib = import('gnome')
-i18n = import('i18n')
-pkgc = import('pkgconfig')
-
 #
 # Directories
 #

@alex-tee
Copy link

alex-tee commented Sep 9, 2023

this is how it fails with docs btw. it also fails on mac with the same error:

[1080/1933] /usr/bin/xsltproc --nonet --stringparam man.output.quietly 1 --stringparam funcsynopsis.style ansi --stringparam man.th.extra1.suppress 1 -o subprojects/appstream/docs/appstreamcli.1 http://docbook.sourceforge.net/release/xsl/current/manpages/docbook.xsl ../subprojects/appstream/docs/xml/man/appstreamcli.1.xml
FAILED: subprojects/appstream/docs/appstreamcli.1 
/usr/bin/xsltproc --nonet --stringparam man.output.quietly 1 --stringparam funcsynopsis.style ansi --stringparam man.th.extra1.suppress 1 -o subprojects/appstream/docs/appstreamcli.1 http://docbook.sourceforge.net/release/xsl/current/manpages/docbook.xsl ../subprojects/appstream/docs/xml/man/appstreamcli.1.xml
I/O error : Attempt to load network entity http://docbook.sourceforge.net/release/xsl/current/manpages/docbook.xsl
warning: failed to load external entity "http://docbook.sourceforge.net/release/xsl/current/manpages/docbook.xsl"
cannot parse http://docbook.sourceforge.net/release/xsl/current/manpages/docbook.xsl

@ximion
Copy link
Owner

ximion commented Sep 10, 2023

this is how it fails with docs btw. it also fails on mac with the same error

Your system is missing docbook schemas. On Debian, they are in the docbook-xsl package, I have no idea how they can be installed on Mac or Windows, but given that it's just data, that shouldn't be too hard. https://packages.debian.org/sid/docbook-xsl

@danyeaw
Copy link
Contributor

danyeaw commented Sep 16, 2023

For building AppStream with MSVC, the dependency on libxmlb is also a blocker since it uses g_autoptr which isn't supported: hughsie/libxmlb#150

@ximion
Copy link
Owner

ximion commented Sep 16, 2023

Well, Appstream itself uses g_autptr everywhere, so I think building with MSVC is a no-go. GCC or Clang are the way to go on Windows, I'm afraid...

@QuLogic
Copy link
Contributor

QuLogic commented Nov 30, 2024

At this point, appstream builds on cross-compiling MinGW at least, but some tests fail in Wine:

  • test_basic:SystemInfo fails because reading the Windows version is not supported.
  • test_misc:RelationSatisfyCheck fails because reading the Windows version is not supported.
  • test_basic:TarExtract fails because I don't have a Windows version of tar; I'm not sure if that should be skipped as well, or ported to something like libarchive.
  • test_pool:PoolEmpty fails trying to save a libxmlb silo. The error looks very similar to hughsie/libxmlb@989cd9c but I can't find what would be holding open the existing file (since it's a new temporary one.)

I've opened #687 fixing the first two.

@QuLogic
Copy link
Contributor

QuLogic commented Nov 30, 2024

test_pool:PoolEmpty fails trying to save a libxmlb silo. The error looks very similar to hughsie/libxmlb@989cd9c but I can't find what would be holding open the existing file (since it's a new temporary one.)

Oh, actually looking at it closer, a lot more tests fail here. They just aren't run since it stopped after the first one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

8 participants