Skip to content

Commit 7af137c

Browse files
joostpvhjoostpvh
committed
Adding the sysdiff migration tool
1 parent c95c596 commit 7af137c

12 files changed

+7385
-4
lines changed

‎OracleSolaris_sysdiff/README.implementation

+53
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
The sysdiff script is intended to create a IPS pkg from non-OS files found in a
2+
solaris10 branded zone.
3+
4+
The script by default excludes some directories deemed unlikely to contain 3rd
5+
party content. The list of directories is read from 'excludeDirs' file in the
6+
script directory. This file is common for both architectures, and can be edited
7+
according to needs.
8+
9+
The SVR4 package database is the foundation against which the zone directory
10+
tree is matched. When the database is ingested, it's checked for 3rd party
11+
packages. This is achieved by matching the encountered packages against the
12+
platform-specific lists of packages provided via the pkglist.{i86pc,sun4v}
13+
files in the script directory. These lists were established based on S10u8 and
14+
S10u11 full+OEM deployment.
15+
16+
Files scanned by the script are sorted into four categories:
17+
- matched files - matching exactly the entries in SVR4 database in the zone
18+
- modified files - files found in the SVR4 database, but not matched exactly
19+
- modified volatile files - as 'modified', but expected to differ - config or
20+
log files etc. (type e or v in SVR4 manifest)
21+
- new files - not found in the SVR4 database
22+
23+
There are some 'new' and 'modified' files appearing in fresh Solaris 10 branded
24+
zone deployments, known to not have been modified after installation from the
25+
distribution media. These files were gathered in platform-specific
26+
known_mod_files/known_new_files.{i86pc,sun4v} files in the script directory,
27+
and are omitted from the generated output. This functionality can be disabled
28+
by either removal of these files, or passing the '-f' option to the 'diff'
29+
subcommand.
30+
31+
The script attempts to guess which files should have the 'preserve' attribute
32+
in the IPS manifest. It's set for the 'volatile' and 'editable' files found in
33+
'contents', and all files modified less than 'preserve age' days ago. By
34+
default 'preserve age' is set to 5 days, but it can be changed by passing the
35+
'--preserve-age' argument to the script.
36+
37+
For the purpose of matching SVR4 owner:group entries it is necessary to convert
38+
the numeric uid:gid stored by the filesystem. This conversion is based on the
39+
passwd/group files from the solaris10 branded zone. If the uid or gid is not
40+
matched against the ingested data, 'getent' in the global zone is run to try
41+
to pull the necessary data from the naming service.
42+
43+
For the IPS package it is necessary to resolve all hardlinks properly. In the
44+
filesystem all hardlinks are equal, and there is not concept of the 'original
45+
file'. Unfortunately a packaging system must distinguish them, so hardliks are
46+
packaged similarly to symlinks - as path + target. This necessitates hardlink
47+
detection and heuristics to determine which path will be considered 'primary'.
48+
The system files are processed first, since there the paths and targets are
49+
defined explicitly by the SVR4 packaging. New files are processed after the
50+
diff is complete, and the first encountered path for given hardlink is treated
51+
as 'file' with all remaining ones being packaged as hardlinks. If this
52+
selection is not valid, the IPS manifest needs to be adjusted by hand.
53+

‎OracleSolaris_sysdiff/README.md

+357
Large diffs are not rendered by default.

‎OracleSolaris_sysdiff/excludeDirs

+26
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
/.SUNWnative
2+
/boot
3+
/dev
4+
/devices
5+
/etc/svc
6+
/media
7+
/mnt
8+
/net
9+
/opt/ocm
10+
/opt/staroffice8
11+
/proc
12+
/root
13+
/rpool
14+
/system
15+
/tmp
16+
/var/adm
17+
/var/cache
18+
/var/core
19+
/var/crash
20+
/var/fm
21+
/var/log
22+
/var/ocm
23+
/var/sadm
24+
/var/spool
25+
/var/svc
26+
/var/tmp

‎OracleSolaris_sysdiff/executionExample

+139
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,139 @@
1+
This file presents an example of the sysdiff execution against a solaris 10
2+
branded zone.
3+
4+
# zoneadm list -civ
5+
...
6+
- s10clean-CPU-mc installed /system/zones/s10clean-CPU-mc solaris10 excl
7+
...
8+
# sysdiff.py -q diff s10clean-CPU-mc
9+
IPS package manifest generated:
10+
/var/tmp/s10clean-CPU-mc-20210319142906/s10clean-CPU-mc.p5m
11+
12+
Please review and adjust it as needed before running the next step.
13+
14+
Please provide the following directory as results-dir for the next steps:
15+
16+
/var/tmp/s10clean-CPU-mc-20210319142906
17+
18+
The next step should be 'sysdiff depresolve'
19+
# sysdiff.py -q depresolve /var/tmp/s10clean-CPU-mc-20210319142906
20+
'sysdiff depresolve' successful, next step is 'sysdiff lint'
21+
# sysdiff.py -q lint --s11.4-repo <URI of the Solaris 11.4 support repo> /var/tmp/s10clean-CPU-mc-20210319142906
22+
command
23+
pkglint -c /var/tmp/s10clean-CPU-mc-20210319142906.cache -r <URI of the Solaris 11.4 support repo> /var/tmp/s10clean-CPU-mc-20210319142906/s10clean-CPU-mc.p5m.dep.res
24+
failed
25+
WARNING pkglint.manifest009.2 pkg.description matches pkg.summary in pkg:/[email protected],5.11
26+
ERROR pkglint.dupaction007 dir action for usr/share/locale/he is reference-counted but has different attributes across 3 duplicates: group: other -> package/pkg system/locale group: root -> s10clean-CPU-mc
27+
! If the 'sysdiff lint' returns an error, it needs to be analyzed and addressed
28+
! before continuing. In this case the issue is that the directory we're trying
29+
! to package has different group than on Solaris 11.4 - root instead of other.
30+
! the issue can be fixed either via removal of the problematic entry from
31+
! the manifest, or by changing the group entry in the manifest to match the OS.
32+
! Whenever the package attempts to deliver files to OS-delivered directories,
33+
! it is recommended to review them carefully to avoid conflicts with OS
34+
! delivered contents in the future.
35+
! The warning can be ignored, or addressed by providing extended description
36+
! for the package.
37+
! Once the manifest is adjusted, the process needs to be restarted from the
38+
! 'sysdiff depresolve' in case the change affected the dependencies of the
39+
! package we're trying to generate.
40+
# sysdiff.py -q depresolve /var/tmp/s10clean-CPU-mc-20210319142906
41+
'sysdiff depresolve' successful, next step is 'sysdiff lint'
42+
# sysdiff.py -q lint --s11.4-repo <URI of the Solaris 11.4 support repo> /var/tmp/s10clean-CPU-mc-20210319142906
43+
'sysdiff lint' successful, next step is 'sysdiff publish'
44+
45+
# zfs create -p rpool/VARSHARE/pkgrepos/sysdiff-repo
46+
# sysdiff.py -q publish -c --target-repo /var/share/pkgrepos/sysdiff-repo/ --publisher sysdiff-publisher /var/tmp/s10clean-CPU-mc-20210319142906
47+
'sysdiff publish' successful, the package can be converted to an archive using 'sysdiff archive'
48+
49+
# pkgrepo list -s /var/share/pkgrepos/sysdiff-repo/
50+
PUBLISHER NAME O VERSION
51+
sysdiff-publisher s10clean-CPU-mc 1.0:20210319T135133Z
52+
53+
Archive creation is an optional step:
54+
55+
# sysdiff.py -q archive --source-repo /var/share/pkgrepos/sysdiff-repo/ /var/tmp/s10clean-CPU-mc-20210319142906
56+
/var/tmp/s10clean-CPU-mc-20210319142906/s10clean-CPU-mc.p5p created successfully
57+
58+
Installation from the repo:
59+
# pkg set-publisher -g /var/share/pkgrepos/sysdiff-repo/ sysdiff-publisher
60+
61+
# zlogin targetzone
62+
# pkg install s10clean-CPU-mc
63+
Packages to install: 1
64+
Create boot environment: No
65+
Create backup boot environment: No
66+
67+
DOWNLOAD PKGS FILES XFER (MB) SPEED
68+
Completed 1/1 152/152 2.2/2.2 --
69+
70+
PHASE ITEMS
71+
Installing new actions 265/265
72+
Updating package state database Done
73+
Updating package cache 0/0
74+
Updating image state Done
75+
Creating fast lookup database Done
76+
Updating package cache 2/2
77+
78+
----
79+
80+
If the manifest needs to be adjusted after testing:
81+
# sysdiff.py -q depresolve /var/tmp/s10clean-CPU-mc-20210319142906
82+
'sysdiff depresolve' successful, next step is 'sysdiff lint'
83+
# sysdiff.py -q lint --s11.4-repo <URI of the Solaris 11.4 support repo> /var/tmp/s10clean-CPU-mc-20210319142906
84+
'sysdiff lint' successful, next step is 'sysdiff publish'
85+
# sysdiff.py -q publish --target-repo /var/share/pkgrepos/sysdiff-repo/ /var/tmp/s10clean-CPU-mc-20210319142906
86+
'sysdiff publish' successful, the package can be converted to an archive using 'sysdiff archive'
87+
# pkgrepo list -s /var/share/pkgrepos/sysdiff-repo/
88+
PUBLISHER NAME O VERSION
89+
sysdiff-publisher s10clean-CPU-mc 1.0:20210319T135508Z
90+
sysdiff-publisher s10clean-CPU-mc 1.0:20210319T135133Z
91+
92+
In the zone:
93+
# pkg refresh
94+
# pkg update s10clean-CPU-mc
95+
Packages to update: 1
96+
Create boot environment: No
97+
Create backup boot environment: Yes
98+
99+
PHASE ITEMS
100+
Removing old actions 1/1
101+
Updating modified actions 1/1
102+
Updating package state database Done
103+
Updating package cache 1/1
104+
Updating image state Done
105+
Creating fast lookup database Done
106+
Updating package cache 2/2
107+
108+
----
109+
110+
Once the package is installed in the target zone, it may happen that one of
111+
the packages it depends on becomes obsolete in the Solaris 11.4, which will
112+
cause the OS update to fail, e.g.:
113+
# pkg update --accept --be-name st_103 [email protected]
114+
...
115+
A 'sync-linked' operation failed for child 'zone:targetzone' with an unexpected
116+
return value of 1 and generated the following output:
117+
118+
pkg sync-linked: Package 's10clean-CPU-mc' must be uninstalled or upgraded if the requested operation is to be performed.
119+
Reject: pkg://sysdiff-publisher/[email protected]
120+
Reason: All acceptable versions of 'require-any' dependencies on frozenset({'runtime/[email protected]'}) are obsolete
121+
122+
There are two possible approaches to addressing this issue. In the short term,
123+
the obsoleted package may be unlocked to allow it to remain installed after the
124+
update:
125+
126+
# zlogin targetzone
127+
# pkg change-facet facet.version-lock.runtime/perl-526=false
128+
129+
This approach may lead to further issues/conflicts in the future, and is
130+
recommended only as a stop-gap solution.
131+
132+
Permanent solution to such an issue is to regenerate the package based on
133+
the original solaris10 branded zone and the preserved 'results-dir'. The
134+
package can be regenerated by running the 'depresolve', 'lint', and 'publish'
135+
steps as during the original creation of the package. Please keep in mind
136+
that the package is timestamped, so it is not necessary to change the version
137+
number during the respin, the updated timestamp is sufficient to make the
138+
package distinct.
139+

‎OracleSolaris_sysdiff/known_mod_files.i86pc

+63
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,63 @@
1+
/export
2+
/lib/svc/method/net-physical
3+
/lib/svc/method/sysidtool-net
4+
/lib/svc/share/net_include.sh
5+
/sbin/dhcpagent
6+
/sbin/dhcpinfo
7+
/sbin/ifconfig
8+
/sbin/umountall
9+
/sbin/zfs
10+
/sbin/zpool
11+
/usr/bin/amd64/adb
12+
/usr/bin/amd64/mdb
13+
/usr/bin/amd64/pfiles
14+
/usr/bin/amd64/prctl
15+
/usr/bin/amd64/truss
16+
/usr/bin/i86/adb
17+
/usr/bin/i86/mdb
18+
/usr/bin/i86/pfiles
19+
/usr/bin/i86/prctl
20+
/usr/bin/i86/truss
21+
/usr/bin/netstat
22+
/usr/bin/priocntl
23+
/usr/lib/autofs/automountd
24+
/usr/lib/class
25+
/usr/lib/class/FSS
26+
/usr/lib/class/FSS/FSSdispadmin
27+
/usr/lib/class/FSS/FSSpriocntl
28+
/usr/lib/class/FX
29+
/usr/lib/class/FX/FXdispadmin
30+
/usr/lib/class/FX/FXpriocntl
31+
/usr/lib/class/IA
32+
/usr/lib/class/IA/IAdispadmin
33+
/usr/lib/class/IA/IApriocntl
34+
/usr/lib/class/RT
35+
/usr/lib/class/RT/RTdispadmin
36+
/usr/lib/class/RT/RTpriocntl
37+
/usr/lib/class/SDC
38+
/usr/lib/class/SDC/SDCdispadmin
39+
/usr/lib/class/SDC/SDCpriocntl
40+
/usr/lib/class/TS
41+
/usr/lib/class/TS/TSdispadmin
42+
/usr/lib/class/TS/TSpriocntl
43+
/usr/lib/dtrace/procfs.d
44+
/usr/lib/fs/autofs/automount
45+
/usr/lib/fs/ufs/quota
46+
/usr/lib/fs/zfs/fstyp
47+
/usr/lib/gss/gssd
48+
/usr/lib/inet/in.mpathd
49+
/usr/lib/inet/in.ndpd
50+
/usr/lib/netsvc/rstat/rpc.rstatd
51+
/usr/lib/security/amd64/pkcs11_kernel.so.1
52+
/usr/lib/security/pkcs11_kernel.so.1
53+
/usr/lib/zfs/pyzfs.py
54+
/usr/sbin/add_drv
55+
/usr/sbin/df
56+
/usr/sbin/if_mpadm
57+
/usr/sbin/in.routed
58+
/usr/sbin/ndd
59+
/usr/sbin/rem_drv
60+
/usr/sbin/snoop
61+
/usr/sbin/umountall
62+
/usr/sbin/zstreamdump
63+
/usr/xpg4/bin/df

0 commit comments

Comments
 (0)