Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Download Git submodules #160

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

alexandre-daubois
Copy link
Contributor

@alexandre-daubois alexandre-daubois commented Dec 23, 2024

Fix #39

As stated in the issue, the .git isn't present in the ZIP archived downloaded from Github. This makes not possible to use git submodules update....

Here is the approach I propose: manually parse the .gitmodules (which isn't a complicated format so it works well), then recursively go through those modules and git clone each of them.

Try this out with : bin/pie build mongodb/mongodb-extension. Note that you may still have an error with this particular extension, but it is not related to this dev. But that's the only extension I found for now that uses submodules 😄

It would be nice to update asgrim/example-pie-extension to add a .gitmodules to test this in the behavioral tests.

Capture d’écran 2024-12-23 à 11 52 36

@asgrim
Copy link
Collaborator

asgrim commented Dec 23, 2024

ping @alcaeus

@alexandre-daubois
Copy link
Contributor Author

Full log of failed build
~/PhpstormProjects/pie git:[git-submodules]
bin/pie build mongodb/mongodb-extension
You are running PHP 8.2.26
Target PHP installation: 8.2.26 nts, on Linux/OSX/etc arm64 (from /opt/homebrew/Cellar/[email protected]/8.2.26/bin/php)
Found package: mongodb/mongodb-extension:1.20.1 which provides ext-mongodb
Extracted mongodb/mongodb-extension:1.20.1 source to: /Users/mac-ADAUBO01/.pie/php8.2_de0add5ce820af84dfd98407ed9d24d8/vendor/mongodb/mongodb-extension
Found .gitmodules file in /Users/mac-ADAUBO01/.pie/php8.2_de0add5ce820af84dfd98407ed9d24d8/vendor/mongodb/mongodb-extension, fetching submodules...
Cloned submodules: https://github.com/mongodb/mongo-c-driver.git, https://github.com/mongodb/libmongocrypt.git
phpize complete.

In Process.php line 270:
                                                                                                                                                                                  
  The command "'./configure'" failed.                                                                                                                                             
                                                                                                                                                                                  
  Exit Code: 1(General error)                                                                                                                                                     
                                                                                                                                                                                  
  Working directory: /Users/mac-ADAUBO01/.pie/php8.2_de0add5ce820af84dfd98407ed9d24d8/vendor/mongodb/mongodb-extension                                                            
                                                                                                                                                                                  
  Output:                                                                                                                                                                         
  ================                                                                                                                                                                
  checking for grep that handles long lines and -e... /usr/bin/grep                                                                                                               
  checking for egrep... /usr/bin/grep -E                                                                                                                                          
  checking for a sed that does not truncate output... /opt/homebrew/opt/gnu-sed/libexec/gnubin/sed                                                                                
  checking for pkg-config... /opt/homebrew/bin/pkg-config                                                                                                                         
  checking pkg-config is at least version 0.9.0... yes                                                                                                                            
  checking for cc... cc                                                                                                                                                           
  checking whether the C compiler works... yes                                                                                                                                    
  checking for C compiler default output file name... a.out                                                                                                                       
  checking for suffix of executables...                                                                                                                                           
  checking whether we are cross compiling... no                                                                                                                                   
  checking for suffix of object files... o                                                                                                                                        
  checking whether the compiler supports GNU C... yes                                                                                                                             
  checking whether cc accepts -g... yes                                                                                                                                           
  checking for cc option to enable C11 features... none needed                                                                                                                    
  checking how to run the C preprocessor... cc -E                                                                                                                                 
  checking for egrep -e... (cached) /usr/bin/grep -E                                                                                                                              
  checking for icc... no                                                                                                                                                          
  checking for suncc... no                                                                                                                                                        
  checking for system library directory... lib                                                                                                                                    
  checking if compiler supports -Wl,-rpath,... yes                                                                                                                                
  checking build system type... aarch64-apple-darwin24.2.0                                                                                                                        
  checking host system type... aarch64-apple-darwin24.2.0                                                                                                                         
  checking target system type... aarch64-apple-darwin24.2.0                                                                                                                       
  checking for PHP prefix... /opt/homebrew/Cellar/[email protected]/8.2.26                                                                                                                  
  checking for PHP includes... -I/opt/homebrew/Cellar/[email protected]/8.2.26/include/php -I/opt/homebrew/Cellar/[email protected]/8.2.26/include/php/main -I/opt/homebrew/Cellar/[email protected]/8.2.26/in  
  clude/php/TSRM -I/opt/homebrew/Cellar/[email protected]/8.2.26/include/php/Zend -I/opt/homebrew/Cellar/[email protected]/8.2.26/include/php/ext -I/opt/homebrew/Cellar/[email protected]/8.2.26/include/php/  
  ext/date/lib                                                                                                                                                                    
  checking for PHP extension directory... /opt/homebrew/Cellar/[email protected]/8.2.26/pecl/20220829                                                                                       
  checking for PHP installed headers prefix... /opt/homebrew/Cellar/[email protected]/8.2.26/include/php                                                                                    
  checking if debug is enabled... no                                                                                                                                              
  checking if zts is enabled... no                                                                                                                                                
  checking for gawk... no                                                                                                                                                         
  checking for nawk... no                                                                                                                                                         
  checking for awk... awk                                                                                                                                                         
  checking if awk is broken... no                                                                                                                                                 
  checking whether to enable MongoDB support... yes, shared                                                                                                                       
  checking for pkg-config... (cached) /opt/homebrew/bin/pkg-config                                                                                                                
  checking pkg-config is at least version 0.9.0... yes                                                                                                                            
  checking PHP version... 8.2.26                                                                                                                                                  
  checking whether to enable developer build flags... no                                                                                                                          
  checking whether to enable code coverage... no                                                                                                                                  
  checking whether to compile against system libraries instead of bundled... no                                                                                                   
  checking whether to use system libbson... no                                                                                                                                    
  checking whether to use system libmongoc... no                                                                                                                                  
  checking whether to enable client-side encryption... auto                                                                                                                       
  checking for gcc... (cached) cc                                                                                                                                                 
  checking whether the compiler supports GNU C... (cached) yes                                                                                                                    
  checking whether cc accepts -g... (cached) yes                                                                                                                                  
  checking for cc option to enable C11 features... (cached) none needed                                                                                                           
  checking for g++... g++                                                                                                                                                         
  checking whether the compiler supports GNU C++... yes                                                                                                                           
  checking whether g++ accepts -g... yes                                                                                                                                          
  checking for g++ option to enable C++11 features... -std=gnu++11                                                                                                                
  checking for an ANSI C-conforming const... yes                                                                                                                                  
  checking for inline... inline                                                                                                                                                   
  checking for typeof syntax and keyword spelling... typeof                                                                                                                       
  checking for __sync_add_and_fetch_4... yes                                                                                                                                      
  checking for __sync_add_and_fetch_8... yes                                                                                                                                      
  checking for stdio.h... yes                                                                                                                                                     
  checking for stdlib.h... yes                                                                                                                                                    
  checking for string.h... yes                                                                                                                                                    
  checking for inttypes.h... yes                                                                                                                                                  
  checking for stdint.h... yes                                                                                                                                                    
  checking for strings.h... yes                                                                                                                                                   
  checking for sys/stat.h... yes                                                                                                                                                  
  checking for sys/types.h... yes                                                                                                                                                 
  checking for unistd.h... yes                                                                                                                                                    
  checking for _Bool... yes                                                                                                                                                       
  checking for stdbool.h that conforms to C99 or later... yes                                                                                                                     
  checking for strings.h... (cached) yes                                                                                                                                          
  checking whether byte ordering is bigendian... no                                                                                                                               
  checking for strnlen... yes                                                                                                                                                     
  checking for reallocf... yes                                                                                                                                                    
  checking for syscall... yes                                                                                                                                                     
  checking for snprintf... yes                                                                                                                                                    
  checking for strlcpy... yes                                                                                                                                                     
  checking for struct timespec... yes                                                                                                                                             
  checking for library containing clock_gettime... none required                                                                                                                  
  checking for library containing floor... none required                                                                                                                          
  checking for gmtime_r... yes                                                                                                                                                    
  checking for rand_r... yes                                                                                                                                                      
  checking for arc4random_buf... yes                                                                                                                                              
  checking whether cc is Clang... yes                                                                                                                                             
  checking whether pthreads work with "-pthread" and "-lpthread"... yes                                                                                                           
  checking whether Clang needs flag to prevent "argument unused" warning when linking with -pthread... no                                                                         
  checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE                                                                                                              
  checking whether more special flags are required for pthreads... no                                                                                                             
  checking for PTHREAD_PRIO_INHERIT... yes                                                                                                                                        
  checking whether PTHREAD_ONCE_INIT needs braces... no                                                                                                                           
  checking whether to enable Snappy for compression... auto                                                                                                                       
  checking whether to enable zlib for compression... auto                                                                                                                         
  checking whether to enable zstd for compression... auto                                                                                                                         
  checking for snappy... no                                                                                                                                                       
  checking for snappy_uncompress in -lsnappy... no                                                                                                                                
  checking for snappy-c.h... no                                                                                                                                                   
  checking for zlib... yes                                                                                                                                                        
  checking for libzstd... yes                                                                                                                                                     
  checking for res_nsearch... yes                                                                                                                                                 
  checking for res_ndestroy... yes                                                                                                                                                
  checking whether to enable SASL for Kerberos authentication... auto                                                                                                             
  checking for libsasl2... no                                                                                                                                                     
  checking for sasl_client_init in -lsasl2... yes                                                                                                                                 
  checking for sasl/sasl.h... yes                                                                                                                                                 
  checking for sasl_client_done in -lsasl2... yes                                                                                                                                 
  checking which SASL library to use... cyrus                                                                                                                                     
  checking whether to enable crypto and TLS... auto                                                                                                                               
  checking deprecated option for OpenSSL library path... auto                                                                                                                     
  checking for cc options needed to detect all undeclared functions... none needed                                                                                                
  checking for openssl >= 1.0.1... yes                                                                                                                                            
  checking whether ASN1_STRING_get0_data is declared... yes                                                                                                                       
  checking which TLS library to use... openssl                                                                                                                                    
  checking whether to use system crypto profile... no                                                                                                                             
  checking deprecated option for whether to use system crypto profile... no                                                                                                       
  checking whether to use bundled or system utf8proc for SCRAM-SHA-256 SASLprep... bundled                                                                                        
  checking whether byte ordering is bigendian... (cached) no                                                                                                                      
  checking for shm_open... yes                                                                                                                                                    
  checking for sched_getcpu... no                                                                                                                                                 
  checking for socklen_t... yes                                                                                                                                                   
  checking for struct sockaddr_storage.ss_family... yes                                                                                                                           
  checking accept ARG2 => struct sockaddr ARG3 => socklen_t ... ok                                                                                                                
  checking for pthread_join using cc -pthread -lpthread... yes                                                                                                                    
  checking whether cc is Clang... (cached) yes                                                                                                                                    
  checking whether Clang needs flag to prevent "argument unused" warning when linking with -pthread... (cached) no                                                                
  checking for joinable pthread attribute... (cached) PTHREAD_CREATE_JOINABLE                                                                                                     
  checking whether more special flags are required for pthreads... (cached) no                                                                                                    
  checking for PTHREAD_PRIO_INHERIT... (cached) yes                                                                                                                               
  checking if weak symbols are supported... yes                                                                                                                                   
  checking which crypto library to use for libmongocrypt... openssl                                                                                                               
  checking whether byte ordering is bigendian... (cached) no                                                                                                                      
  checking for a sed that does not truncate output... /opt/homebrew/opt/gnu-sed/libexec/gnubin/sed                                                                                
  checking for ld used by cc... /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld                                                         
  checking if the linker (/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld) is GNU ld... no                                              
  checking for /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld option to reload object files... -r                                      
  checking for BSD-compatible nm... /usr/bin/nm -B                                                                                                                                
  checking whether ln -s works... yes                                                                                                                                             
  checking how to recognize dependent libraries... pass_all                                                                                                                       
  checking for dlfcn.h... yes                                                                                                                                                     
  checking the maximum length of command line arguments... 786432                                                                                                                 
  checking command to parse /usr/bin/nm -B output from cc object... ok                                                                                                            
  checking for objdir... .libs                                                                                                                                                    
  checking for ar... ar                                                                                                                                                           
  checking for ranlib... ranlib                                                                                                                                                   
  checking for strip... strip                                                                                                                                                     
  checking for dsymutil... dsymutil                                                                                                                                               
  checking for nmedit... nmedit                                                                                                                                                   
  checking for -single_module linker flag... yes                                                                                                                                  
  checking for -exported_symbols_list linker flag... yes                                                                                                                          
  checking if cc supports -fno-rtti -fno-exceptions... yes                                                                                                                        
  checking for cc option to produce PIC... -fno-common                                                                                                                            
  checking if cc PIC flag -fno-common works... yes                                                                                                                                
  checking if cc static flag -static works... no                                                                                                                                  
  checking if cc supports -c -o file.o... yes                                                                                                                                     
  checking whether the cc linker (/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld) supports shared libraries... yes                     
  checking dynamic linker characteristics... darwin24.2.0 dyld                                                                                                                    
  checking how to hardcode library paths into programs... immediate                                                                                                               
  checking whether stripping libraries is possible... yes                                                                                                                         
  checking if libtool supports shared libraries... yes                                                                                                                            
  checking whether to build shared libraries... yes                                                                                                                               
  checking whether to build static libraries... no                                                                                                                                
                                                                                                                                                                                  
  creating libtool                                                                                                                                                                
  appending configuration tag "CXX" to libtool                                                                                                                                    
  checking for ld used by g++ -std=gnu++11... /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld                                           
  checking if the linker (/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld) is GNU ld... no                                              
  checking whether the g++ -std=gnu++11 linker (/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld) supports shared libraries... yes       
  checking for g++ -std=gnu++11 option to produce PIC... -fno-common                                                                                                              
  checking if g++ -std=gnu++11 PIC flag -fno-common works... yes                                                                                                                  
  checking if g++ -std=gnu++11 static flag -static works... no                                                                                                                    
  checking if g++ -std=gnu++11 supports -c -o file.o... yes                                                                                                                       
  checking whether the g++ -std=gnu++11 linker (/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld) supports shared libraries... yes       
  checking dynamic linker characteristics... darwin24.2.0 dyld                                                                                                                    
  (cached) (cached) checking how to hardcode library paths into programs... immediate                                                                                             
  configure: patching config.h.in                                                                                                                                                 
  configure: creating ./config.status                                                                                                                                             
                                                                                                                                                                                  
  mongodb was configured with the following options:                                                                                                                              
                                                                                                                                                                                  
  Build configuration:                                                                                                                                                            
    CC                                               : cc                                                                                                                         
    CFLAGS                                           : -g -O2                                                                                                                     
    CPPFLAGS                                         :  -DHAVE_CONFIG_H                                                                                                           
    Extra flags                                      :                                                                                                                            
    Developer flags (slow)                           :                                                                                                                            
    Code Coverage flags (extra slow)                 :                                                                                                                            
    libmongoc                                        : Bundled (1.28.1)                                                                                                           
    libbson                                          : Bundled (1.28.1)                                                                                                           
    libmongocrypt                                    : Bundled (1.11.0)                                                                                                           
    LDFLAGS                                          :                                                                                                                            
    EXTRA_LDFLAGS                                    :                                                                                                                            
    MONGODB_SHARED_LIBADD                            : -Wl,-rpath,/opt/homebrew/Cellar/openssl@3/3.4.0/lib -L/opt/homebrew/Cellar/openssl@3/3.4.0/lib -lsasl2 -lresolv -Wl,-rpat  
  h,/opt/homebrew/opt/zstd/lib -L/opt/homebrew/opt/zstd/lib  -lpthread -pthread -lz -lzstd -lssl -lcrypto -lpthread -pthread                                                      
                                                                                                                                                                                  
  Please submit bugreports at:                                                                                                                                                    
    https://jira.mongodb.org/browse/PHPC                                                                                                                                          
                                                                                                                                                                                  
                                                                                                                                                                                  
                                                                                                                                                                                  
                                                                                                                                                                                  
  Error Output:                                                                                                                                                                   
  ================                                                                                                                                                                
  ld: warning: -single_module is obsolete                                                                                                                                         
  config.status: error: cannot find input file: '/Users/mac-ADAUBO01/.pie/php8.2_de0add5ce820af84dfd98407ed9d24d8/vendor/mongodb/mongodb-extension/src/libmongoc/src/common/comm  
  on-config.h.in'                                                                                                                                                                 
                                                                                                                                                                                  

build [-j|--make-parallel-jobs MAKE-PARALLEL-JOBS] [--with-phpize-path WITH-PHPIZE-PATH] [--with-php-config WITH-PHP-CONFIG] [--with-php-path WITH-PHP-PATH] [--enable-mongodb-coverage ENABLE-MONGODB-COVERAGE] [--enable-mongodb-crypto-system-profile ENABLE-MONGODB-CRYPTO-SYSTEM-PROFILE] [--enable-mongodb-developer-flags ENABLE-MONGODB-DEVELOPER-FLAGS] [--with-mongodb-client-side-encryption WITH-MONGODB-CLIENT-SIDE-ENCRYPTION] [--with-mongodb-sasl WITH-MONGODB-SASL] [--with-mongodb-snappy WITH-MONGODB-SNAPPY] [--with-mongodb-ssl WITH-MONGODB-SSL] [--with-mongodb-system-libs WITH-MONGODB-SYSTEM-LIBS] [--with-mongodb-zlib WITH-MONGODB-ZLIB] [--with-mongodb-zstd WITH-MONGODB-ZSTD] [--] <requested-package-and-version>

@GromNaN
Copy link
Member

GromNaN commented Dec 23, 2024

You can't just take the last commit from the main branch, you need the exact commit that was committed. Maybe we could specify the commit in the .gitmodules file.

@alexandre-daubois
Copy link
Contributor Author

It seems you can provide .gitmodules only a branch name, not a commit hash. 😕 Note that the error I have in the failed build seems present as well without this PR.

$downloadedPackage->extractedSourcePath,
));

$git = GitBinaryPath::fromGitBinaryPath('/opt/homebrew/bin/git');

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This path is specific to macOS package manager, it won't work on other systems.

I'm not even sure it is needed to specify git path as it should be in $PATH of the user

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah yes indeed, it was for test purposes to validate the implementation. Thanks for spotting it 👍

@asgrim asgrim marked this pull request as draft December 24, 2024 10:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Investigate if git submodules can be included
4 participants