Skip to content

Commit

Permalink
Reviewer comments and final review (#1)
Browse files Browse the repository at this point in the history
* 371: Clarify Monorepo

* 380 and 387 : Typos and section references

* 385: whitespace does not look nice

* 377: definition of OEM

* Fix autoformatter's whitespace madness

* 389: improve wording

* 390: font is Libertine in all figures

* ignore *.ts

* 393, 378, 381, 388, 389, 390, 379: worked further on findings (JM and KG)

* worked on references.bib

* 382: Reformulate sentence

* 383: express figure 2, get rid of some warnings.

* 384: devops handbook

* 376: short abstract

* 394: Scaling within introduction rephrased

* 396: feature model rephrased

* 386: refrase unit test demand paragraph

* 423: earlier intro of space and time dimension in variation

* 375: explain fear

* 395: restructure conclusion and focus on future work

* 375: Corrected typo

* 374: modularization

* Final review by Jochen and Karsten. DONE

Co-authored-by: Maletschek, Jochen (SD-RM) <[email protected]>
  • Loading branch information
xxthunder and jomal77 authored Jul 8, 2022
1 parent 4a30106 commit 52ae25e
Show file tree
Hide file tree
Showing 17 changed files with 474 additions and 417 deletions.
Binary file not shown.
2 changes: 2 additions & 0 deletions paper/.gitignore
Original file line number Diff line number Diff line change
@@ -1,10 +1,12 @@
/texmfs
/*.bak
/*.aux
/*.log
/*.out
/*.bbl
/*.blg
/*.fdb_latexmk
/*.fls
/*.ts
/*.synctex.gz
/paper.pdf
16 changes: 8 additions & 8 deletions paper/challenge.tex
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
\section{Challenge}
\section{Challenge}\label{challenge}

Our migration method is supposed to be a generic solution for a migration from a
polyrepo project landscape to a monorepo SPL, but our circumstances might have
some influence on the ideas that we share in this paper. This section will
explain the challenges we had, both coming from the products' nature and its
industry and also from our corporate processes and strategies.
Our migration method is supposed to be a generic solution, but our company's
circumstances might have some influence on the ideas that we share in this
paper. This section will explain the challenges we had, both coming from the
products' nature and its industry and also from our corporate processes and
strategies.

The projects that we migrated are in the area of electric cars for premium
automobile manufacturers. The software is written in (Embedded) C and Matlab and
Expand All @@ -29,7 +29,7 @@ \section{Challenge}
Dimensions repository. Dimensions is a Source Code Management (SCM) system
similar to Revision Control System (RCS) used at the Marquardt GmbH for source
code administration and project documentation. Each individual source code
repository is a self-contained ready to build project, including all required
repository is a self-contained ready-to-build project, including all required
tools and libraries to build the product binaries, like compilers, MSYS and GNU
Make. The C code was validated with Jenkins nightly builds. Those Jenkins
scripts were located in separate repositories, not part of the project's
Expand All @@ -41,7 +41,7 @@ \section{Challenge}
already existing project (`clone-and-own'). This approach had several benefits:
the setup time was short and because a mature project was taken as basis, the
project's software was stable already and debugging and adapting was possible.
There was less work for generic parts and only hardware and customer specific
There was less work for generic parts and only hardware and customer-specific
modifications were required. However, this approach also caused a lot of
disadvantages. By simply cloning an existing project there was not only a copy
of functionality available in the new project, but also a copy of all its flaws.
Expand Down
46 changes: 23 additions & 23 deletions paper/conclusion.tex
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
\section{Conclusion and Future Work}
\section{Conclusion and Future Work}\label{conclusion}

This work provides an iterative and incremental migration strategy towards SPL
for the software of a real-world product family of an automotive supplier. We
Expand All @@ -15,40 +15,40 @@ \section{Conclusion and Future Work}
\item established a stable SPL with variants buildable at any time,
\item reduced workload of developers by reducing the number of repositories,
\item increased collaboration beyond project borders and
\item integrated all software sources of a product family into one git
\item integrated all software sources of a product family into one Git
repository.
\end{itemize}

We implemented a build system with modern CMake fully supporting the concept of
SPL variants.

At the submission date's point of time we managed to proof about 50\% of our
concept. The build system prototype required two persons full-time for six
weeks, the implementation for migration step one and two required two persons
full-time for three more months. Over a time frame of two months we
incrementally added ten variants, while each import to the SPL was really fast
due to the automated importing mechanism. We already started with
concept. We implemented a build system with modern CMake fully supporting the
concept of SPL variants. The build system prototype required two persons
full-time for six weeks, the implementation for migration step one and two
required two persons full-time for three more months. Over a time frame of two
months we incrementally added ten variants, while each import to the SPL was
really fast due to the automated importing mechanism. We already started with
\textit{Modularization} of software components, but only converted a few
components and begun with removing duplicates. Recently we focussed more on
components and begun with removing duplicates. Recently we focused more on
automation than on the SPL migration, as we saw more advantages there in the
long run. The automation for CI/CD is helpful for the migration as well.
Nevertheless, the build system requires some more features to be able to handle
our migration step three and we plan to implement them in the coming months, in
parallel to the \textit{Modularization} step of the software components.

Additionally to the build system implementation and the actual migration of the
embedded source code, it is required for us to qualify tools which have an
impact on the created binaries or the processes to build them. This tool
Nevertheless, the build system requires more features, e.g., a package manager
for dependency handling and a unit test framework for Test Driven Development
(TDD) to be able to support our migration steps two and three and we plan to
implement them in the coming months, in parallel to the \textit{Modularization}
of the software components. Additionally it is required to qualify tools which
have an impact on the created binaries or the processes to build them. This tool
qualification is requested by the ISO 26262 standard for safety reasons. We must
take care that CMake, Ninja and KConfig are properly qualified. On the other
hand, proprietary software might come with safety evaluations already. This
should be kept in mind, when deciding for your tool landscape.
should be kept in mind, when deciding for a future tool landscape.

Besides all the discussed technical challenges, there were also social obstacles
on the way to SPL. Fear was probably the strongest emotion we faced. At the
beginning we were certain that all required build system features were
implemented and we always had a backup plan to go back to the original
repositories. There was no real threat and still no one had enough confidence to
start the migration. Afterwards we do not see a reason for hesitation, but
on the way to introduce an SPL.\@ Fear of change was probably the strongest
emotion we faced. We feared the introduction of our SPL approach, although we
were certain that all required build system features were implemented and we
always had a backup plan to go back to the original repositories. On the other
hand the development team had a lot of doubts to adapt fast enough to new tools
and processes. There was no real threat and still no one had enough confidence
to start the migration. Afterwards we do not see a reason for hesitation, but
sometimes it simply needs someone brave in the team or someone pushing you. Our
conclusion is: Just do it! And if you break it, make sure you can fix it fast!
Binary file added paper/fonts/LinLibertine_Rah.ttf
Binary file not shown.
1 change: 1 addition & 0 deletions paper/fonts/source.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
LateX class "acmart" uses font Libertine. Got it from here: https://libertine-fonts.org/
Binary file modified paper/images/aspl_one_pagers_concept.pptx
Binary file not shown.
Binary file modified paper/images/migration-steps-overview.pdf
Binary file not shown.
Binary file modified paper/images/migration_steps.pdf
Binary file not shown.
Binary file modified paper/images/migration_steps.pptx
Binary file not shown.
2 changes: 1 addition & 1 deletion paper/images/step2-modularization-flow.drawio
Original file line number Diff line number Diff line change
@@ -1 +1 @@
<mxfile host="Electron" modified="2022-06-10T14:12:46.079Z" agent="5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) draw.io/19.0.3 Chrome/102.0.5005.63 Electron/19.0.3 Safari/537.36" version="19.0.3" etag="4Wro6nL2rmszLiauxm7u" type="device"><diagram id="C5RBs43oDa-KdzZeNtuy" name="Page-1">7VxZc9o6FP41zLQPZLyC80homt5u0zb3Nu1TR9jCqJEtV5ZZ+usr2RLY2BgTNnObzISgY1lIR+f7ziKTjjkM5ncURJMPxIO4Y2jevGO+6hiGYZv8VQgWUmD0MoFPkZeJ9JXgHv2GUqhJaYI8GBc6MkIwQ1FR6JIwhC4ryAClZFbsNia4+KkR8GFJcO8CXJY+II9NMqlj9FfyNxD5E/XJeu86uxIA1VmuJJ4Aj8xyIvO2Yw4pISx7F8yHEAvdKb08/LN4wO8fe3dvP8e/wH837/79+LWbDfZ6l1uWS6AwZE8e+iuZv/Ws5F33mzebzP0PP35c064uFxuzhVIY9Lj+ZJNQNiE+CQG+XUlvKElCD4phNd5a9XlPSMSFOhf+hIwtpDGAhBEumrAAy6twjtg3cfuVLVvfc1dezeXIaWOhGiGji9xNovk9f211W9oq3PcJUhRABqkUNlSoVHxMEurCGi1KA2GA+pDVadvOOgoV56xT7tcdJHyOdME7UIgBQ9OiCQOJBH/Zb7Xb/I3c8B02X057CnCiVgcoAlwlhjYo2cVsghi8j0CqiRnniuKegjjK4DtGc2EbO+p4CimD81qlKEayrOwWSUh9uYxZDt1SNMkBW8kOrsUlPTxDaA8IKdewFUL9VkFIWW8VhG5aCyGz1zII6SVVPUNoZwgZTSHktApCRg2Ehq2FkOW0C0I9sz0Q0q4cpx5FvLEOhqcia0xC9hoECAvBG4inkCEXyAty4roh20OCifhAD45BgtkR3JjdEISGcWgQprcOKAWLXIeIoJDFuZE/CUEumFozY2XHrzf0N43a/vxNNoOVIS+XsoeTtUsU4ZIgIiFM19YDgWCBcBRH6YZqZdEaMmJGyeMyHTSKNs3TvEj0C+a+SIivxpjM3Amg7CrgJoO6HnGTILWWmyoyOl7YqxU0r19XMI5TZpze0RjHemacNjBO/7IYx9IugXH6z4zDs4S2MY5x4YzTMuZwLos57HomaAlzOM/MwZOjljGH02sPc9QXGPakiKfGNAemFsUYW6nF0Y9CLVu5wLLXTC2bqbxrzdp2o4X6cnDImaDaSvRO3rvYtVay2ubCJq/2fMM2eyCepBadkkfKLMqGOoappT9ntIa+dWhr2K8yVS5NDSeExFB8GP/F0AeumM6S4cWF0OOvAZmKXkhIOC3wl4lo+zDkIYErrgQRhoKa+TpIyAUeotBlhC6uSvazIiJ9A5HnzOlYnG5Y1wUEmXqZ03X7lCUvs0XhYNOqcRGuxja8HhqI54v49gNiOai652tg5f3HGEUx3I6SUp24iLFjgUjXi4HRuavGzt95/N+uPMxqevZpma1CpZp3DpU8m0k15KFpwap6vxLxlMzNCLiPfmo/XTdT60BM1x+9MGy7Y/B5aPk3L7McCaMQdhUexA36dTRPL6lx+Ttf/P3C90g40aXDzXvmGeIYE08QuQnPxLTU607FUdMIYcQWau4jqkZTEq6ddEFKuo97rrKwSlM8lR+3raZ+3NzfjurPeQ8UG1cUXjaHuk+NpA/NAWZTDjj4I0QbyrT9NU9lHC1hqg/jjmcU50mGGu+0qpmcu15vVxvCUatoSkk51/KJEhfGcZmlPWEggtYF6wMx9nqPfAHuMNU1D7ooFnlbNd8XPQIGI4hvlp5P2Zg07JPGIju4iV7RTfR6dtlN6BVuYik8fLxRPga+Fcn2heUA60+O9JwzJwF2GWxlpYbeQDyMzVsuBnGM3KIuiznBvkVMlR/sSfCbfHu/rc798Gl3NafrRpHTbccuDpEtqeTcywNdFweytGZRwq5OSK93Qlv7m6c4+rHL/BSSytjmvfAIAjUwRr/BKL2krZETRj53L68wHAtICHrhEMIDKR4RxkhwbldTSyfyCx1yeZ1lYbw5UyqA7WntXVX7V7eQ8TiG+0attQXsipQ4jkBYmROXs1ge6niJyzJdxVl5mIzXstkXFPAUN8t005EjSn5yfb/MZa3ZZ25NW7XtvvIiYxhLmaJiAa0i1bWqYph1Gjtcwc2uJIQTFdxOccS0py2cJylzmpbgnHadUDltLMHJs680FdO8JMJ8/xmMFYmxCYo7VWdmHg/fplnRjqYDxJyl46u/tC5nGec+X1NPaPx/DwfadQ7QmIRU4b0tJFQOer5AHsBAAeowCUZplFIKYKbZtzzKNZm/LzCxqs7STxuYqEQut4mD9K588UwDU4BwGtGnU0iD/DHyEyplmY3HS0ofQ8CSlMwD+b1ylIaqWGzn877b9in3PQ6GX8y3tw8PdPTz8+Db3fARfqz49tUCliF56Ulr41y06a5vfmja6Re2uHRu0mDneHP1XwWy1HT1rxnM2z8=</diagram></mxfile>
<mxfile host="Electron" modified="2022-07-08T13:11:22.551Z" agent="5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) draw.io/19.0.3 Chrome/102.0.5005.63 Electron/19.0.3 Safari/537.36" etag="ilkoquzPVNKv4u-L_zYM" version="19.0.3" type="device"><diagram id="C5RBs43oDa-KdzZeNtuy" name="Page-1">7Vxbd5s4EP41Pmf3wTlcbfKYpEm6adJtk92mfeqRQbbVCERBxHZ/fSWQbC4yxjGxSZq+BA0CI818M9/MQHvmmT+/jEA4vSEexD1D8+Y9813PMHTdsdkfLllkEsfQM8EkQp6YtBLcoV9QCDUhTZAH48JESgimKCwKXRIE0KUFGYgiMitOGxNc/NUQTGBFcOcCXJXeI49O5SqGK/l7iCZT+cv64Dg74wM5WawkngKPzHIi87xnnkWE0OzIn59BzDdP7sv9P4t7fP0wuLz6HP8E/59++O/jl352s4ttLlkuIYIBffKtv5D5lWclH/pfvdl0Prn5/v046utisTFdyA2DHts/MSQRnZIJCQA+X0lPI5IEHuS31dhoNeeakJAJdSb8ASldCGMACSVMNKU+FmfhHNGv/PIjW4y+5c68m4s7p4OFHAQ0WuQu4sNv+XOry9JR4bpPMEI+pDASwjEJ6AXwEeazrlGQzNkeXKMRjCgK+BIb7rjQTEySyIU12ywsiIJoAmmdOgTYuA5y5isUegkJW0S0YBMiiAFFj0UbBwIqk+W8lTmwA2ERW1iHeOxHgBO5OhAhwLbE0E4qhjObIgrvQpDuxIx5k6LSQRxm+B6jOTeetpXwyK6D89pdE2cNWyxMuLRjMZyt/MNAiKY51yBlrW/z0sG8gfA5QSijz0YQDjsFQmneKhCevlwQmsOOgVCv7OUbCNsHodEUhE6nQGjUgPDs5YLQOu4WCAdmd0CoHTlOPQ7ZoAynp2JzowXwCWIBuiHGZwQT/sMeHIME02eImHZDtBpG22hNLz2JIrDITQgJCmicu/MnLsgRO8cqmLNhmXmT3Dx/qNXONw1tp/mmXpjPDrIVrgCz3Kod+IJd8VUu8UMSwHTvBsDn7igYxWFqMFpVVEJgTCPysEye9SJ2WFIc8nn+fMLrB0djTGbuFET0yGcmifoecRM/tcZTlVc8XA5gDgqq0RWuz3H26Pok63xTWy1rtLumNudNbQ14xrBjanMG3SEa9Wy/JWbwVGLSMqOQRGEjo3D0Z2EUihBdpACWXTK57EnFVSWr2y5a11d5ApJqU2Etei9PSu1aa1mpuaDklc7XqNkD8TS17NTLpC5I2lDPMLX03yGyxabmMrTaNpfd0sVqvsgcfwrEETuY8IMbxHw4hWzWOCI++4PhBLj8GZcBhK+c8P2MiJe4tHhqIBj/xfKOFftaOSx9TUTImdvhqJhlHdlKnpwPD7qtSESd54oPpvFS4sM6xBubIL93LDsNsdx+MrkblqsM746tgVYNBGMUxnAz0ir1nyJODwZE3exWNch5a052pxZkNW2fiJpLV+Arn1sRij30WLCuwc+EpCEauA+T1I76bratJ/xxJ6O/DJsFKvYcWv7g7yyzw0w5fYkLfoF+ZKRn5G1XYbrEBG6ZzlzKfodDjSGNv+XgJrE6xC9l6dMXpV1ZEAnSDY8QGCGM6KJuJaNo09p2JTVlqHSE5NjmHklOfXOrpdxDUQ9fn0o8NVPZN22R7myz32v93Q11xmqVGqdD49ky1noS/HxW09FstLEpSOJ04HaIZZUsRa9vV5Qta+DsoV0hN1URn0NlMFOEJV0Zlm5Y0oxZEPrFtphHpHK88bj98ljLAy/gT1qeUajkytgUro1MTyzdetBFMX9CdTQrxjsMRhCfLuO5hIXA4kGo4xZBcFAyMFsRAuUrifkQqJd9XHv0sNotOw+815fbWaVWp8T2wZI7W4X78q4H3gl/BZiNXAziGLnFzS7mem1VymX+t2MQW0dwhpsYzqGiVvt1F3WY0UtdcVu+Yi5vkS2pQmCqNyq9vWJpzZjQtnFUH2wXR8vzzeEe4qhddWQBUfK3ax5COHpgzGLjKD2llbwYRhMWj95hOOaQ4G4GuQCfCPGIUEr8rsSmWvciPisQy+wtuzDNPacE2o5W35eNJnkJGY9juCtDr22GKChVHIJAyaqq9OkWJnG6bZv6D9k9N2br2ubg+aLJjVV6q3z58k+e3Jh7rZ/aSvzvqX66j7ZlSybRMN9sOTA7TSuqTream85LqKj65JF7Li8JMYtcFMZ8Jzixf6k1VY/Ry3RJLolgqlYWPWprxH9sZdVSed69to/lu0yvt3PVzSZVY5cqu0JdcalVxnYLGeVi1mBoQeKzbWUHZFxkYFmLRaQub6wrG6u6KntlXTIpzenyJL0qX03UwCNAWNBrEGeJyhhNkkjIMlNfRi1tDAFNUsfvi0+0ESfyIeZa/YPVXypx28OGrr8N9cf+2a15dX5/H41+fD75enn2AD8qvhRawCpAX2MeXqfR3dNw81hdhNlJg3furK852uThBg/uvYsf76/+vVJo0J0SEnPsZR2DnAt+pWptrK6mFrD+naPj44JaDTluXa3b1Zurzc2iU13Lm8RXY9u+9KOkI83x1fCj5qoacn5S5SalbOe3rNUfTm1d8rWMNUWWlku+y1Ky+kuvnUu4Sgsd/NkW6hzSQtcpfGcLNaznsVC7/lvEbS2UDVf/k002ffX/AZnnvwE=</diagram></mxfile>
Binary file modified paper/images/step2-modularization-flow.pdf
Binary file not shown.
53 changes: 27 additions & 26 deletions paper/introduction.tex
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
\section{Introduction}
\section{Introduction}\label{introduction}

Development of product families rather than individual products is a goal for
many engineering companies. Product Line Engineering (PLE) is a widely used
Expand All @@ -16,39 +16,40 @@ \section{Introduction}
Lines (SPL)~\cite{confsplc2000}. An SPL increases reusability~\cite{Clem02a} and
therefore increases quality and decreases development costs of software by
building up shared core assets and customer-specific functionality. With this
reuse in place, a business will be able to do larger scaling, with higher number
of projects the development costs will drastically decrease compared to
traditional engineering methods and the time-to-market will improve due to less
development effort~\cite{confsplcAzanzaMD21}. Thus an SPL is one possible
solution to win against competitors.
reuse in place, a business will be able to do larger scaling. A higher number of
projects will drastically decrease the development costs compared to traditional
engineering methods and the time-to-market will improve due to less development
effort~\cite{confsplcAzanzaMD21}. Thus an SPL is one possible solution to win
against competitors.

But even after years of research, source code migrations to SPL are still
complicated and there are no bullet proof concepts available that are applicable
to actively running projects with tight release plans. Much research has been
done on product line architectures~\cite{Svahnberg1999EPLA,
confsplcTomashchukLJ21} and system engineering \cite{confsplcSchaferBAKR21},
confsplcTomashchukLJ21} and system engineering~\cite{confsplcSchaferBAKR21},
about feature model migrations~\cite{ncstrlustuttgartfiINPROC200185,
confsplcGrunerBKR20, confsplcDuszynskiDB19, confsplcFritschAR20}, SPL
evolution~\cite{journalssmrQuintonVRBGS21, Svahnberg1999ESPL, Eise02b,
kconfigKernel} and safety aspects~\cite{confsplcWolschke0SAM19} of SPLs. On the
other hand, \cite{confoopslaHetrickKM06} and \cite{confsplcAbbasJLESS20} have
written about the overall process, general benefits and a change of mindset,
but not specifically about source code. Contrary to \cite{confsplcRubinCC13},
cloned variants are no option for us anymore. The authors of
\cite{Krueger2001SMC} reported different migration strategies more than 20 years
ago already: (1) proactive, (2) extractive and (3) reactive, but no work
described a mixture of extractive and reactive migrations, which we needed.
Inspired by the work of \cite{confsplcJepsenDB07} we aim for a strategy with
which we are able to migrate multiple repositories to a monorepo, allowing
developers and software product managers to proceed with their own pace and
according to the project plan without risking its deadlines and always being
able to do a step backwards if neccessary.
other hand, Hetrick et al.~\cite{confoopslaHetrickKM06} and Abbas et
al.~\cite{confsplcAbbasJLESS20} have written about the overall process, general
benefits and a change of mindset, but not specifically about source code.
Contrary to Rubin et al.~\cite{confsplcRubinCC13}, cloned variants are no option
for us anymore. Krueger~\cite{Krueger2001SMC} reported different migration
strategies more than 20 years ago already: (1) proactive, (2) extractive and (3)
reactive, but no work described a mixture of extractive and reactive migrations,
which we need for existing and upcoming customer projects. Inspired by the work
of Jepsen et al.~\cite{confsplcJepsenDB07} we aim for a strategy with which we
are able to migrate multiple independent repositories to a scalable SPL,
allowing developers and software product managers to proceed with their own pace
and according to the project plan without risking its deadlines and always being
able to do a step backwards if necessary.

In this paper we propose an iterative and incremental migration strategy in
three steps. We structured the paper in the following way: Section two stating
the challenges we faced in our projects and industry to explain our view on the
problem and why standard solutions did not work for us, section three explaining
our three step solution of the source code migration in detail, presenting
infrastructure and build system ideas and section four presenting our
conclusion, an overview on where we stand right now and giving a forecast on
future work.
three steps. We structured the paper in the following way:
Section~\ref{challenge} stating the challenges we faced in our projects and
industry to explain our view on the problem and why standard solutions did not
work for us, section~\ref{solution} explaining our three step solution of the
source code migration in detail, presenting infrastructure and build system
ideas and section~\ref{conclusion} presenting our conclusion, an overview on
where we stand right now and giving a forecast on future work.
Loading

0 comments on commit 52ae25e

Please sign in to comment.