Skip to content

Commit

Permalink
qapi: Generate separate .h, .c for each module
Browse files Browse the repository at this point in the history 
Our qapi-schema.json is composed of modules connected by include
directives, but the generated code is monolithic all the same: one
qapi-types.h with all the types, one qapi-visit.h with all the
visitors, and so forth.  These monolithic headers get included all
over the place.  In my "build everything" tree, adding a QAPI type
recompiles about 4800 out of 5100 objects.

We wouldn't write such monolithic headers by hand.  It stands to
reason that we shouldn't generate them, either.

Split up generated qapi-types.h to mirror the schema's modular
structure: one header per module.  Name the main module's header
qapi-types.h, and sub-module D/B.json's header D/qapi-types-B.h.

Mirror the schema's includes in the headers, so that qapi-types.h gets
you everything exactly as before.  If you need less, you can include
one or more of the sub-module headers.  To be exploited shortly.

Split up qapi-types.c, qapi-visit.h, qapi-visit.c, qmp-commands.h,
qmp-commands.c, qapi-event.h, qapi-event.c the same way.
qmp-introspect.h, qmp-introspect.c and qapi.texi remain monolithic.

The split of qmp-commands.c duplicates static helper function
qmp_marshal_output_str() in qapi-commands-char.c and
qapi-commands-misc.c.  This happens when commands returning the same
type occur in multiple modules.  Not worth avoiding.

Since I'm going to rename qapi-event.[ch] to qapi-events.[ch], and
qmp-commands.[ch] to qapi-commands.[ch], name the shards that way
already, to reduce churn.  This requires temporary hacks in
commands.py and events.py.  Similarly, c_name() must temporarily
be taught to munge '/' in common.py.  They'll go away with the rename.

Signed-off-by: Markus Armbruster <[email protected]>
Message-Id: <[email protected]>
Reviewed-by: Eric Blake <[email protected]>
[eblake: declare a dummy variable in each .c file, to shut up OSX
toolchain warnings about empty .o files, including hacking c_name()]
Signed-off-by: Eric Blake <[email protected]>
  • Loading branch information
@ebblake
Markus Armbruster authored and ebblake committed Mar 2, 2018
1 parent 3da7a41 commit 252dc31
Show file tree
Hide file tree
Showing 6 changed files with 310 additions and 22 deletions.
60 changes: 60 additions & 0 deletions .gitignore
View file
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,67 @@
/qapi-gen-timestamp
/qapi-builtin-types.[ch]
/qapi-builtin-visit.[ch]
/qapi/qapi-commands-block-core.[ch]
/qapi/qapi-commands-block.[ch]
/qapi/qapi-commands-char.[ch]
/qapi/qapi-commands-common.[ch]
/qapi/qapi-commands-crypto.[ch]
/qapi/qapi-commands-introspect.[ch]
/qapi/qapi-commands-migration.[ch]
/qapi/qapi-commands-net.[ch]
/qapi/qapi-commands-rocker.[ch]
/qapi/qapi-commands-run-state.[ch]
/qapi/qapi-commands-sockets.[ch]
/qapi/qapi-commands-tpm.[ch]
/qapi/qapi-commands-trace.[ch]
/qapi/qapi-commands-transaction.[ch]
/qapi/qapi-commands-ui.[ch]
/qapi/qapi-events-block-core.[ch]
/qapi/qapi-events-block.[ch]
/qapi/qapi-events-char.[ch]
/qapi/qapi-events-common.[ch]
/qapi/qapi-events-crypto.[ch]
/qapi/qapi-events-introspect.[ch]
/qapi/qapi-events-migration.[ch]
/qapi/qapi-events-net.[ch]
/qapi/qapi-events-rocker.[ch]
/qapi/qapi-events-run-state.[ch]
/qapi/qapi-events-sockets.[ch]
/qapi/qapi-events-tpm.[ch]
/qapi/qapi-events-trace.[ch]
/qapi/qapi-events-transaction.[ch]
/qapi/qapi-events-ui.[ch]
/qapi/qapi-types-block-core.[ch]
/qapi/qapi-types-block.[ch]
/qapi/qapi-types-char.[ch]
/qapi/qapi-types-common.[ch]
/qapi/qapi-types-crypto.[ch]
/qapi/qapi-types-introspect.[ch]
/qapi/qapi-types-migration.[ch]
/qapi/qapi-types-net.[ch]
/qapi/qapi-types-rocker.[ch]
/qapi/qapi-types-run-state.[ch]
/qapi/qapi-types-sockets.[ch]
/qapi/qapi-types-tpm.[ch]
/qapi/qapi-types-trace.[ch]
/qapi/qapi-types-transaction.[ch]
/qapi/qapi-types-ui.[ch]
/qapi-types.[ch]
/qapi/qapi-visit-block-core.[ch]
/qapi/qapi-visit-block.[ch]
/qapi/qapi-visit-char.[ch]
/qapi/qapi-visit-common.[ch]
/qapi/qapi-visit-crypto.[ch]
/qapi/qapi-visit-introspect.[ch]
/qapi/qapi-visit-migration.[ch]
/qapi/qapi-visit-net.[ch]
/qapi/qapi-visit-rocker.[ch]
/qapi/qapi-visit-run-state.[ch]
/qapi/qapi-visit-sockets.[ch]
/qapi/qapi-visit-tpm.[ch]
/qapi/qapi-visit-trace.[ch]
/qapi/qapi-visit-transaction.[ch]
/qapi/qapi-visit-ui.[ch]
/qapi-visit.[ch]
/qapi-event.[ch]
/qapi-doc.texi
Expand Down
120 changes: 120 additions & 0 deletions Makefile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -92,10 +92,70 @@ include $(SRC_PATH)/rules.mak
GENERATED_FILES = qemu-version.h config-host.h qemu-options.def
GENERATED_FILES += qapi-builtin-types.h qapi-builtin-types.c
GENERATED_FILES += qapi-types.h qapi-types.c
GENERATED_FILES += qapi/qapi-types-block-core.h qapi/qapi-types-block-core.c
GENERATED_FILES += qapi/qapi-types-block.h qapi/qapi-types-block.c
GENERATED_FILES += qapi/qapi-types-char.h qapi/qapi-types-char.c
GENERATED_FILES += qapi/qapi-types-common.h qapi/qapi-types-common.c
GENERATED_FILES += qapi/qapi-types-crypto.h qapi/qapi-types-crypto.c
GENERATED_FILES += qapi/qapi-types-introspect.h qapi/qapi-types-introspect.c
GENERATED_FILES += qapi/qapi-types-migration.h qapi/qapi-types-migration.c
GENERATED_FILES += qapi/qapi-types-net.h qapi/qapi-types-net.c
GENERATED_FILES += qapi/qapi-types-rocker.h qapi/qapi-types-rocker.c
GENERATED_FILES += qapi/qapi-types-run-state.h qapi/qapi-types-run-state.c
GENERATED_FILES += qapi/qapi-types-sockets.h qapi/qapi-types-sockets.c
GENERATED_FILES += qapi/qapi-types-tpm.h qapi/qapi-types-tpm.c
GENERATED_FILES += qapi/qapi-types-trace.h qapi/qapi-types-trace.c
GENERATED_FILES += qapi/qapi-types-transaction.h qapi/qapi-types-transaction.c
GENERATED_FILES += qapi/qapi-types-ui.h qapi/qapi-types-ui.c
GENERATED_FILES += qapi-builtin-visit.h qapi-builtin-visit.c
GENERATED_FILES += qapi-visit.h qapi-visit.c
GENERATED_FILES += qapi/qapi-visit-block-core.h qapi/qapi-visit-block-core.c
GENERATED_FILES += qapi/qapi-visit-block.h qapi/qapi-visit-block.c
GENERATED_FILES += qapi/qapi-visit-char.h qapi/qapi-visit-char.c
GENERATED_FILES += qapi/qapi-visit-common.h qapi/qapi-visit-common.c
GENERATED_FILES += qapi/qapi-visit-crypto.h qapi/qapi-visit-crypto.c
GENERATED_FILES += qapi/qapi-visit-introspect.h qapi/qapi-visit-introspect.c
GENERATED_FILES += qapi/qapi-visit-migration.h qapi/qapi-visit-migration.c
GENERATED_FILES += qapi/qapi-visit-net.h qapi/qapi-visit-net.c
GENERATED_FILES += qapi/qapi-visit-rocker.h qapi/qapi-visit-rocker.c
GENERATED_FILES += qapi/qapi-visit-run-state.h qapi/qapi-visit-run-state.c
GENERATED_FILES += qapi/qapi-visit-sockets.h qapi/qapi-visit-sockets.c
GENERATED_FILES += qapi/qapi-visit-tpm.h qapi/qapi-visit-tpm.c
GENERATED_FILES += qapi/qapi-visit-trace.h qapi/qapi-visit-trace.c
GENERATED_FILES += qapi/qapi-visit-transaction.h qapi/qapi-visit-transaction.c
GENERATED_FILES += qapi/qapi-visit-ui.h qapi/qapi-visit-ui.c
GENERATED_FILES += qmp-commands.h qmp-commands.c
GENERATED_FILES += qapi/qapi-commands-block-core.h qapi/qapi-commands-block-core.c
GENERATED_FILES += qapi/qapi-commands-block.h qapi/qapi-commands-block.c
GENERATED_FILES += qapi/qapi-commands-char.h qapi/qapi-commands-char.c
GENERATED_FILES += qapi/qapi-commands-common.h qapi/qapi-commands-common.c
GENERATED_FILES += qapi/qapi-commands-crypto.h qapi/qapi-commands-crypto.c
GENERATED_FILES += qapi/qapi-commands-introspect.h qapi/qapi-commands-introspect.c
GENERATED_FILES += qapi/qapi-commands-migration.h qapi/qapi-commands-migration.c
GENERATED_FILES += qapi/qapi-commands-net.h qapi/qapi-commands-net.c
GENERATED_FILES += qapi/qapi-commands-rocker.h qapi/qapi-commands-rocker.c
GENERATED_FILES += qapi/qapi-commands-run-state.h qapi/qapi-commands-run-state.c
GENERATED_FILES += qapi/qapi-commands-sockets.h qapi/qapi-commands-sockets.c
GENERATED_FILES += qapi/qapi-commands-tpm.h qapi/qapi-commands-tpm.c
GENERATED_FILES += qapi/qapi-commands-trace.h qapi/qapi-commands-trace.c
GENERATED_FILES += qapi/qapi-commands-transaction.h qapi/qapi-commands-transaction.c
GENERATED_FILES += qapi/qapi-commands-ui.h qapi/qapi-commands-ui.c
GENERATED_FILES += qapi-event.h qapi-event.c
GENERATED_FILES += qapi/qapi-events-block-core.h qapi/qapi-events-block-core.c
GENERATED_FILES += qapi/qapi-events-block.h qapi/qapi-events-block.c
GENERATED_FILES += qapi/qapi-events-char.h qapi/qapi-events-char.c
GENERATED_FILES += qapi/qapi-events-common.h qapi/qapi-events-common.c
GENERATED_FILES += qapi/qapi-events-crypto.h qapi/qapi-events-crypto.c
GENERATED_FILES += qapi/qapi-events-introspect.h qapi/qapi-events-introspect.c
GENERATED_FILES += qapi/qapi-events-migration.h qapi/qapi-events-migration.c
GENERATED_FILES += qapi/qapi-events-net.h qapi/qapi-events-net.c
GENERATED_FILES += qapi/qapi-events-rocker.h qapi/qapi-events-rocker.c
GENERATED_FILES += qapi/qapi-events-run-state.h qapi/qapi-events-run-state.c
GENERATED_FILES += qapi/qapi-events-sockets.h qapi/qapi-events-sockets.c
GENERATED_FILES += qapi/qapi-events-tpm.h qapi/qapi-events-tpm.c
GENERATED_FILES += qapi/qapi-events-trace.h qapi/qapi-events-trace.c
GENERATED_FILES += qapi/qapi-events-transaction.h qapi/qapi-events-transaction.c
GENERATED_FILES += qapi/qapi-events-ui.h qapi/qapi-events-ui.c
GENERATED_FILES += qmp-introspect.c qmp-introspect.h
GENERATED_FILES += qapi-doc.texi

Expand Down Expand Up @@ -524,10 +584,70 @@ qapi-modules = $(SRC_PATH)/qapi-schema.json $(SRC_PATH)/qapi/common.json \

qapi-builtin-types.c qapi-builtin-types.h \
qapi-types.c qapi-types.h \
qapi/qapi-types-block-core.c qapi/qapi-types-block-core.h \
qapi/qapi-types-block.c qapi/qapi-types-block.h \
qapi/qapi-types-char.c qapi/qapi-types-char.h \
qapi/qapi-types-common.c qapi/qapi-types-common.h \
qapi/qapi-types-crypto.c qapi/qapi-types-crypto.h \
qapi/qapi-types-introspect.c qapi/qapi-types-introspect.h \
qapi/qapi-types-migration.c qapi/qapi-types-migration.h \
qapi/qapi-types-net.c qapi/qapi-types-net.h \
qapi/qapi-types-rocker.c qapi/qapi-types-rocker.h \
qapi/qapi-types-run-state.c qapi/qapi-types-run-state.h \
qapi/qapi-types-sockets.c qapi/qapi-types-sockets.h \
qapi/qapi-types-tpm.c qapi/qapi-types-tpm.h \
qapi/qapi-types-trace.c qapi/qapi-types-trace.h \
qapi/qapi-types-transaction.c qapi/qapi-types-transaction.h \
qapi/qapi-types-ui.c qapi/qapi-types-ui.h \
qapi-builtin-visit.c qapi-builtin-visit.h \
qapi-visit.c qapi-visit.h \
qapi/qapi-visit-block-core.c qapi/qapi-visit-block-core.h \
qapi/qapi-visit-block.c qapi/qapi-visit-block.h \
qapi/qapi-visit-char.c qapi/qapi-visit-char.h \
qapi/qapi-visit-common.c qapi/qapi-visit-common.h \
qapi/qapi-visit-crypto.c qapi/qapi-visit-crypto.h \
qapi/qapi-visit-introspect.c qapi/qapi-visit-introspect.h \
qapi/qapi-visit-migration.c qapi/qapi-visit-migration.h \
qapi/qapi-visit-net.c qapi/qapi-visit-net.h \
qapi/qapi-visit-rocker.c qapi/qapi-visit-rocker.h \
qapi/qapi-visit-run-state.c qapi/qapi-visit-run-state.h \
qapi/qapi-visit-sockets.c qapi/qapi-visit-sockets.h \
qapi/qapi-visit-tpm.c qapi/qapi-visit-tpm.h \
qapi/qapi-visit-trace.c qapi/qapi-visit-trace.h \
qapi/qapi-visit-transaction.c qapi/qapi-visit-transaction.h \
qapi/qapi-visit-ui.c qapi/qapi-visit-ui.h \
qmp-commands.h qmp-commands.c \
qapi/qapi-commands-block-core.c qapi/qapi-commands-block-core.h \
qapi/qapi-commands-block.c qapi/qapi-commands-block.h \
qapi/qapi-commands-char.c qapi/qapi-commands-char.h \
qapi/qapi-commands-common.c qapi/qapi-commands-common.h \
qapi/qapi-commands-crypto.c qapi/qapi-commands-crypto.h \
qapi/qapi-commands-introspect.c qapi/qapi-commands-introspect.h \
qapi/qapi-commands-migration.c qapi/qapi-commands-migration.h \
qapi/qapi-commands-net.c qapi/qapi-commands-net.h \
qapi/qapi-commands-rocker.c qapi/qapi-commands-rocker.h \
qapi/qapi-commands-run-state.c qapi/qapi-commands-run-state.h \
qapi/qapi-commands-sockets.c qapi/qapi-commands-sockets.h \
qapi/qapi-commands-tpm.c qapi/qapi-commands-tpm.h \
qapi/qapi-commands-trace.c qapi/qapi-commands-trace.h \
qapi/qapi-commands-transaction.c qapi/qapi-commands-transaction.h \
qapi/qapi-commands-ui.c qapi/qapi-commands-ui.h \
qapi-event.c qapi-event.h \
qapi/qapi-events-block-core.c qapi/qapi-events-block-core.h \
qapi/qapi-events-block.c qapi/qapi-events-block.h \
qapi/qapi-events-char.c qapi/qapi-events-char.h \
qapi/qapi-events-common.c qapi/qapi-events-common.h \
qapi/qapi-events-crypto.c qapi/qapi-events-crypto.h \
qapi/qapi-events-introspect.c qapi/qapi-events-introspect.h \
qapi/qapi-events-migration.c qapi/qapi-events-migration.h \
qapi/qapi-events-net.c qapi/qapi-events-net.h \
qapi/qapi-events-rocker.c qapi/qapi-events-rocker.h \
qapi/qapi-events-run-state.c qapi/qapi-events-run-state.h \
qapi/qapi-events-sockets.c qapi/qapi-events-sockets.h \
qapi/qapi-events-tpm.c qapi/qapi-events-tpm.h \
qapi/qapi-events-trace.c qapi/qapi-events-trace.h \
qapi/qapi-events-transaction.c qapi/qapi-events-transaction.h \
qapi/qapi-events-ui.c qapi/qapi-events-ui.h \
qmp-introspect.h qmp-introspect.c \
qapi-doc.texi: \
qapi-gen-timestamp ;
Expand Down
65 changes: 64 additions & 1 deletion Makefile.objs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,56 @@
stub-obj-y = stubs/ crypto/
util-obj-y = util/ qobject/ qapi/
util-obj-y += qapi-builtin-types.o
util-obj-y += qapi-types.o
util-obj-y += qapi/qapi-types-block-core.o
util-obj-y += qapi/qapi-types-block.o
util-obj-y += qapi/qapi-types-char.o
util-obj-y += qapi/qapi-types-common.o
util-obj-y += qapi/qapi-types-crypto.o
util-obj-y += qapi/qapi-types-introspect.o
util-obj-y += qapi/qapi-types-migration.o
util-obj-y += qapi/qapi-types-net.o
util-obj-y += qapi/qapi-types-rocker.o
util-obj-y += qapi/qapi-types-run-state.o
util-obj-y += qapi/qapi-types-sockets.o
util-obj-y += qapi/qapi-types-tpm.o
util-obj-y += qapi/qapi-types-trace.o
util-obj-y += qapi/qapi-types-transaction.o
util-obj-y += qapi/qapi-types-ui.o
util-obj-y += qapi-builtin-visit.o
util-obj-y += qmp-introspect.o qapi-types.o qapi-visit.o qapi-event.o
util-obj-y += qapi-visit.o
util-obj-y += qapi/qapi-visit-block-core.o
util-obj-y += qapi/qapi-visit-block.o
util-obj-y += qapi/qapi-visit-char.o
util-obj-y += qapi/qapi-visit-common.o
util-obj-y += qapi/qapi-visit-crypto.o
util-obj-y += qapi/qapi-visit-introspect.o
util-obj-y += qapi/qapi-visit-migration.o
util-obj-y += qapi/qapi-visit-net.o
util-obj-y += qapi/qapi-visit-rocker.o
util-obj-y += qapi/qapi-visit-run-state.o
util-obj-y += qapi/qapi-visit-sockets.o
util-obj-y += qapi/qapi-visit-tpm.o
util-obj-y += qapi/qapi-visit-trace.o
util-obj-y += qapi/qapi-visit-transaction.o
util-obj-y += qapi/qapi-visit-ui.o
util-obj-y += qapi-event.o
util-obj-y += qapi/qapi-events-block-core.o
util-obj-y += qapi/qapi-events-block.o
util-obj-y += qapi/qapi-events-char.o
util-obj-y += qapi/qapi-events-common.o
util-obj-y += qapi/qapi-events-crypto.o
util-obj-y += qapi/qapi-events-introspect.o
util-obj-y += qapi/qapi-events-migration.o
util-obj-y += qapi/qapi-events-net.o
util-obj-y += qapi/qapi-events-rocker.o
util-obj-y += qapi/qapi-events-run-state.o
util-obj-y += qapi/qapi-events-sockets.o
util-obj-y += qapi/qapi-events-tpm.o
util-obj-y += qapi/qapi-events-trace.o
util-obj-y += qapi/qapi-events-transaction.o
util-obj-y += qapi/qapi-events-ui.o
util-obj-y += qmp-introspect.o

chardev-obj-y = chardev/

Expand Down Expand Up @@ -81,6 +129,21 @@ common-obj-$(CONFIG_FDT) += device_tree.o
# qapi

common-obj-y += qmp-commands.o
common-obj-y += qapi/qapi-commands-block-core.o
common-obj-y += qapi/qapi-commands-block.o
common-obj-y += qapi/qapi-commands-char.o
common-obj-y += qapi/qapi-commands-common.o
common-obj-y += qapi/qapi-commands-crypto.o
common-obj-y += qapi/qapi-commands-introspect.o
common-obj-y += qapi/qapi-commands-migration.o
common-obj-y += qapi/qapi-commands-net.o
common-obj-y += qapi/qapi-commands-rocker.o
common-obj-y += qapi/qapi-commands-run-state.o
common-obj-y += qapi/qapi-commands-sockets.o
common-obj-y += qapi/qapi-commands-tpm.o
common-obj-y += qapi/qapi-commands-trace.o
common-obj-y += qapi/qapi-commands-transaction.o
common-obj-y += qapi/qapi-commands-ui.o
common-obj-y += qmp-introspect.o
common-obj-y += qmp.o hmp.o
endif
Expand Down
35 changes: 24 additions & 11 deletions scripts/qapi/commands.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -223,14 +223,24 @@ def gen_registry(registry, prefix):
return ret


class QAPISchemaGenCommandVisitor(QAPISchemaMonolithicCVisitor):
class QAPISchemaGenCommandVisitor(QAPISchemaModularCVisitor):

def __init__(self, prefix):
QAPISchemaMonolithicCVisitor.__init__(
self, prefix, 'qmp-commands',
QAPISchemaModularCVisitor.__init__(
self, prefix, 'qapi-commands',
' * Schema-defined QAPI/QMP commands', __doc__)
self._regy = ''
self._visited_ret_types = set()
self._visited_ret_types = {}

# Temporary HACK:
def _module_basename(self, what, name):
basename = QAPISchemaModularCVisitor._module_basename(self, what, name)
if name == self._main_module:
return re.sub(r'qapi-commands', 'qmp-commands', basename)
return basename

def _begin_module(self, name):
self._visited_ret_types[self._genc] = set()
self._genc.add(mcgen('''
#include "qemu/osdep.h"
#include "qemu-common.h"
Expand All @@ -246,26 +256,29 @@ def __init__(self, prefix):
#include "%(prefix)sqmp-commands.h"
''',
prefix=prefix))
prefix=self._prefix))
self._genh.add(mcgen('''
#include "%(prefix)sqapi-types.h"
#include "qapi/qmp/dispatch.h"
void %(c_prefix)sqmp_init_marshal(QmpCommandList *cmds);
''',
prefix=prefix,
c_prefix=c_name(prefix, protect=False)))
prefix=self._prefix))

def visit_end(self):
self._genc.add(gen_registry(self._regy, self._prefix))
(genc, genh) = self._module[self._main_module]
genh.add(mcgen('''
void %(c_prefix)sqmp_init_marshal(QmpCommandList *cmds);
''',
c_prefix=c_name(self._prefix, protect=False)))
genc.add(gen_registry(self._regy, self._prefix))

def visit_command(self, name, info, arg_type, ret_type,
gen, success_response, boxed):
if not gen:
return
self._genh.add(gen_command_decl(name, arg_type, boxed, ret_type))
if ret_type and ret_type not in self._visited_ret_types:
self._visited_ret_types.add(ret_type)
if ret_type and ret_type not in self._visited_ret_types[self._genc]:
self._visited_ret_types[self._genc].add(ret_type)
self._genc.add(gen_marshal_output(ret_type))
self._genh.add(gen_marshal_decl(name))
self._genc.add(gen_marshal(name, arg_type, boxed, ret_type))
Expand Down
Loading

0 comments on commit 252dc31

Please sign in to comment.