[NPU] Model marshalling without weights copies (#31939)

### Details:
* Extends the [OV StreamSerializer &
XmlSerializer](#31639)
in order to allow passing an `ov::Model` through the driver without
copying its weights into a separate buffer.
* The purpose of this PR is to reduce memory consumption by avoiding
weights duplications.
* This feature will be disabled by default for the CiD interface (at
least for a while). The changes are first meant to be integrated in the
upcoming CiP interface.
* The implementation followed and adapted the sample provided in [this
PR](#31969).
* Two config options introduced: `intel_npu::use_base_model_serializer`
- a switch between the old & new serialization algorithms, and
`intel_npu::serialization_weights_size_threshold` - controls which
weights are copied into a separate buffer and which ones have only
metadata (memory location & size) stored as runtime information. More
concretely, weights smaller than this value will be copied.
* `vcl_serializer.hpp` is meant to contain all operations required to
prepare an `ov::Model` as food for the VCL interface. This implies using
the new/old model serializer, I/O & config serialization.
`xml_serializer.hpp` is a more generic, weightless (no weights copies
unless `serialization_weights_size_threshold` is used) implementation of
the OV serializer.
* Roughly how this works:
* The plugin passes through all `ov::Constant` nodes and places weights
metadata (`intel_npu::WeightsPointerAttribute`) as runtime information
on the nodes that have buffers smaller than
`serialization_weights_size_threshold`.
* The new `intel_npu::StreamSerialize` is called which uses the
`intel_npu::XmlSerializer` for serializing the model. Note that
`StreamSerialize` uses a slightly different format within the buffer
(metadata containing offsets & sizes, custom data, weights & the XML
graph), see `ov::pass::StreamSerialize` for details.
* `intel_npu::XmlSerializer` will not write weights into its dedicated
buffer if the `WeightsPointerAttribute` is found within the current
`ov::Constant` node. Instead, weights metadata will be written as
runtime information by calling the visit method corresponding to the
attribute.
* The deserializer will be able to distinguish between the two cases
(weights copied vs. weights stored as metadata) by looking for this
attribute in the serialized buffer.
* See the ticket for some performance reports.

## Related PRs
* [Sample for extending the serialization
algorithm](#31969)
* [The PR that made the OV serializer
extensible](#31639)

### Tickets:
 - *CVS-173711*

Author: razvanapetroaie

src/core/xml_util/CMakeLists.txt

@@ -19,7 +19,7 @@ source_group("include" FILES ${PUBLIC_HEADERS})
 add_library(${TARGET_NAME} STATIC ${LIBRARY_SRC} ${PUBLIC_HEADERS})
 
 add_library(openvino::xml_util ALIAS ${TARGET_NAME})
-set_target_properties(${TARGET_NAME} PROPERTIES EXPORT_NAME openvino_xml_util)
+set_target_properties(${TARGET_NAME} PROPERTIES EXPORT_NAME xml_util)
 
 target_include_directories(${TARGET_NAME} PUBLIC
     $<BUILD_INTERFACE:${TARGET_INCLUDE_DIR}>

src/plugins/intel_npu/src/al/include/intel_npu/config/options.hpp

@@ -1426,4 +1426,18 @@ struct USE_BASE_MODEL_SERIALIZER final : OptionBase<USE_BASE_MODEL_SERIALIZER, b
     }
 };
 
+struct SERIALIZATION_WEIGHTS_SIZE_THRESHOLD final : OptionBase<SERIALIZATION_WEIGHTS_SIZE_THRESHOLD, size_t> {
+    static std::string_view key() {
+        return ov::intel_npu::serialization_weights_size_threshold.name();
+    }
+
+    static size_t defaultValue() {
+        return 0;
+    }
+
+    static OptionMode mode() {
+        return OptionMode::RunTime;
+    }
+};
+
 }  // namespace intel_npu

src/plugins/intel_npu/src/al/include/intel_npu/npu_private_properties.hpp

@@ -357,12 +357,21 @@ static constexpr ov::Property<bool> weightless_blob{"NPU_WEIGHTLESS_BLOB"};
  *
  * The base serializer is the OV implementation of the "XmlSerializer" without any extensions. All weights are copied in
  * a separate buffer. By turning this off, the NPU extension of the serializer is enabled. This allows optimizing the
- * process by avoiding copies into a separate weights buffer. However, this solution may be less reliable.
- *
- * @note This option doesn't actually do anything right now, it has been registered in advance.
+ * process by reducing the amount of weights that will be copied in a separate buffer. However, this solution may be
+ * less reliable.
  */
 static constexpr ov::Property<bool> use_base_model_serializer{"NPU_USE_BASE_MODEL_SERIALIZER"};
 
+/**
+ * @brief [Only for NPU Plugin]
+ * Type: size_t. Default is 0.
+ *
+ * Effective only if "use_base_model_serializer" is set to false. All "ov::Constant" buffers smaller than this value
+ * (byte size) will be copied in a separate buffer. The rest of the weights will be reconstructed at deserialization
+ * time using buffer pointers.
+ */
+static constexpr ov::Property<size_t> serialization_weights_size_threshold{"NPU_SERIALIZATION_WEIGHTS_SIZE_THRESHOLD"};
+
 /**
  * @brief [Experimental, only for NPU Plugin]
  * Type: integer.

src/plugins/intel_npu/src/al/include/intel_npu/weights_pointer_attribute.hpp

@@ -0,0 +1,46 @@
+// Copyright (C) 2025 Intel Corporation.
+// SPDX-License-Identifier: Apache-2.0
+//
+
+#pragma once
+
+#include <string_view>
+
+#include "openvino/core/runtime_attribute.hpp"
+
+namespace intel_npu {
+
+/**
+ * @brief Attribute containing the memory address of a weights buffer and the size of the buffer in bytes.
+ * @details Used as part of the serialization/deserialization algorithms in order to allow processing models without
+ * copying weights.
+ */
+class WeightsPointerAttribute : public ov::RuntimeAttribute {
+public:
+    OPENVINO_RTTI("WeightsPointerAttribute", "0", RuntimeAttribute);
+
+    WeightsPointerAttribute() = delete;
+
+    WeightsPointerAttribute(const void* pointer, const size_t size)
+        : memory_pointer(reinterpret_cast<size_t>(pointer)),
+          byte_size(size) {}
+
+    /**
+     * @note The names of the attributes have been kept short in order to save some memory (there may be a lot of
+     * "ov::Constant" nodes in a model). While deserializing, the name of the attribute ("WeightsPointerAttribute") is
+     * also used as part of identification in order to avoid collision.
+     */
+    static constexpr const std::string_view POINTER_KEY = "mp";
+    static constexpr const std::string_view BYTE_SIZE_KEY = "ms";
+
+    bool visit_attributes(ov::AttributeVisitor& visitor) override {
+        visitor.on_attribute(POINTER_KEY.data(), memory_pointer);
+        visitor.on_attribute(BYTE_SIZE_KEY.data(), byte_size);
+        return true;
+    }
+
+    size_t memory_pointer;
+    size_t byte_size;
+};
+
+}  // namespace intel_npu

src/plugins/intel_npu/src/common/include/intel_npu/common/icompiler_adapter.hpp

@@ -4,14 +4,15 @@
 
 #pragma once
 
+#include "intel_npu/common/filtered_config.hpp"
 #include "intel_npu/common/igraph.hpp"
 
 namespace intel_npu {
 
 class ICompilerAdapter {
 public:
     virtual std::shared_ptr<IGraph> compile(const std::shared_ptr<const ov::Model>& model,
-                                            const Config& config) const = 0;
+                                            const FilteredConfig& config) const = 0;
 
     /**
      * @brief Compiles the model, weights separation enabled.
@@ -27,7 +28,8 @@ class ICompilerAdapter {
      * "icompiler.hpp".
      * @return A "WeightlessGraph" type of object.
      */
-    virtual std::shared_ptr<IGraph> compileWS(const std::shared_ptr<ov::Model>& model, const Config& config) const = 0;
+    virtual std::shared_ptr<IGraph> compileWS(const std::shared_ptr<ov::Model>& model,
+                                              const FilteredConfig& config) const = 0;
 
     /**
      * @brief Parses the provided binary objects and returns a wrapper over the resulted L0 handles. The model may also
@@ -44,11 +46,12 @@ class ICompilerAdapter {
      */
     virtual std::shared_ptr<IGraph> parse(
         ov::Tensor mainBlob,
-        const Config& config,
+        const FilteredConfig& config,
         std::optional<std::vector<ov::Tensor>> initBlobs = std::nullopt,
         const std::optional<std::shared_ptr<const ov::Model>>& model = std::nullopt) const = 0;
 
-    virtual ov::SupportedOpsMap query(const std::shared_ptr<const ov::Model>& model, const Config& config) const = 0;
+    virtual ov::SupportedOpsMap query(const std::shared_ptr<const ov::Model>& model,
+                                      const FilteredConfig& config) const = 0;
     virtual uint32_t get_version() const = 0;
     virtual std::vector<std::string> get_supported_options() const = 0;
     virtual bool is_option_supported(std::string optname) const = 0;

src/plugins/intel_npu/src/compiler_adapter/CMakeLists.txt

@@ -25,6 +25,7 @@ target_link_libraries(${TARGET_NAME}
     PRIVATE
         openvino::npu_al
         openvino::npu_common
+        openvino::xml_util
 )
 
 #

src/plugins/intel_npu/src/compiler_adapter/include/custom_stream_buffer.hpp

@@ -75,6 +75,11 @@ class writer_streambuf final : public std::streambuf {
         }
     }
 
+    pos_type seekpos(pos_type pos, std::ios_base::openmode which) override {
+        writeIt = startIt + pos;
+        return pos;
+    }
+
     OutputIt startIt;
     OutputIt writeIt;
 };

src/plugins/intel_npu/src/compiler_adapter/include/driver_compiler_adapter.hpp

@@ -10,6 +10,7 @@
 #include "intel_npu/config/config.hpp"
 #include "intel_npu/utils/logger/logger.hpp"
 #include "intel_npu/utils/zero/zero_init.hpp"
+#include "vcl_serializer.hpp"
 #include "ze_graph_ext_wrappers.hpp"
 
 namespace intel_npu {
@@ -18,17 +19,20 @@ class DriverCompilerAdapter final : public ICompilerAdapter {
 public:
     DriverCompilerAdapter(const std::shared_ptr<ZeroInitStructsHolder>& zeroInitStruct);
 
-    std::shared_ptr<IGraph> compile(const std::shared_ptr<const ov::Model>& model, const Config& config) const override;
+    std::shared_ptr<IGraph> compile(const std::shared_ptr<const ov::Model>& model,
+                                    const FilteredConfig& config) const override;
 
-    std::shared_ptr<IGraph> compileWS(const std::shared_ptr<ov::Model>& model, const Config& config) const override;
+    std::shared_ptr<IGraph> compileWS(const std::shared_ptr<ov::Model>& model,
+                                      const FilteredConfig& config) const override;
 
     std::shared_ptr<IGraph> parse(
         ov::Tensor mainBlob,
-        const Config& config,
+        const FilteredConfig& config,
         std::optional<std::vector<ov::Tensor>> initBlobs = std::nullopt,
         const std::optional<std::shared_ptr<const ov::Model>>& model = std::nullopt) const override;
 
-    ov::SupportedOpsMap query(const std::shared_ptr<const ov::Model>& model, const Config& config) const override;
+    ov::SupportedOpsMap query(const std::shared_ptr<const ov::Model>& model,
+                              const FilteredConfig& config) const override;
 
     std::vector<std::string> get_supported_options() const override;
 

src/plugins/intel_npu/src/compiler_adapter/include/ir_serializer.hpp

@@ -19,17 +19,20 @@ class PluginCompilerAdapter final : public ICompilerAdapter {
 public:
     PluginCompilerAdapter(const std::shared_ptr<ZeroInitStructsHolder>& zeroInitStruct);
 
-    std::shared_ptr<IGraph> compile(const std::shared_ptr<const ov::Model>& model, const Config& config) const override;
+    std::shared_ptr<IGraph> compile(const std::shared_ptr<const ov::Model>& model,
+                                    const FilteredConfig& config) const override;
 
-    std::shared_ptr<IGraph> compileWS(const std::shared_ptr<ov::Model>& model, const Config& config) const override;
+    std::shared_ptr<IGraph> compileWS(const std::shared_ptr<ov::Model>& model,
+                                      const FilteredConfig& config) const override;
 
     std::shared_ptr<IGraph> parse(
         ov::Tensor mainBlob,
-        const Config& config,
+        const FilteredConfig& config,
         std::optional<std::vector<ov::Tensor>> initBlobs = std::nullopt,
         const std::optional<std::shared_ptr<const ov::Model>>& model = std::nullopt) const override;
 
-    ov::SupportedOpsMap query(const std::shared_ptr<const ov::Model>& model, const Config& config) const override;
+    ov::SupportedOpsMap query(const std::shared_ptr<const ov::Model>& model,
+                              const FilteredConfig& config) const override;
 
     std::vector<std::string> get_supported_options() const override;