Feature: Merge `Expr` and `GenExpr` #1114

Zeroto521 · 2025-11-18T03:27:45Z

Close to #1074. This is a breaking change. We can release the 5.7.0 version first.

Changed the type of _lhs and _rhs attributes in the ExprCons class from unspecified to 'object' to clarify their expected type and improve type safety.

Replaces the __init__ method with __cinit__ in the Variable class and updates the argument type to SCIP_VAR*.

Updated references from 'terms' to 'children' for Expr objects throughout Model methods to reflect changes in the Expr API. This ensures compatibility with the updated data structure and avoids errors when accessing expression terms.

Introduces _to_nodes methods for Expr, PolynomialExpr, and UnaryExpr to convert expressions into node lists for SCIP construction. Refactors Model's constraint creation to use the new node format, simplifying and clarifying the mapping from expression trees to SCIP nonlinear constraints.

Changed Expr from a Cython cdef class to a standard Python class for improved compatibility and maintainability. Removed cdef public dict children, as attribute is now managed in Python.

This reverts commit 28e6673.

Converted SumExpr, ProdExpr, and PowExpr from cdef classes to regular Python classes for improved compatibility and maintainability.

Copilot

Pull Request Overview

This PR merges the Expr and GenExpr classes into a unified expression system. The refactoring replaces the previous dual-system (polynomial Expr and general GenExpr) with a single hierarchy based on Expr, using specialized subclasses like PolynomialExpr, SumExpr, ProdExpr, and various function expressions (ExpExpr, LogExpr, etc.).

Key changes:

Unified expression representation with children replacing terms
Variable class no longer inherits from Expr
Simplified expression tree structure with improved type system
Refactored constraint creation methods to use the new expression system

Reviewed Changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
src/pyscipopt/scip.pyi	Updated `Variable` class to remove inheritance from `Expr`
src/pyscipopt/scip.pxi	Modified `Variable` class structure and added operator overloading methods to delegate to expression system; refactored constraint creation methods to use `children` instead of `terms`
src/pyscipopt/scip.pxd	Updated `Variable` class declaration to remove `Expr` inheritance
src/pyscipopt/propagator.pxi	Simplified variable creation by removing unnecessary temporary variable
src/pyscipopt/expr.pxi	Complete rewrite of expression system replacing `Expr`/`GenExpr` duality with unified class hierarchy

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

src/pyscipopt/expr.pxi

Co-authored-by: Copilot <[email protected]>

Deleted the definition of the Expr class, which was not used in the code. This helps clean up the codebase and improves maintainability.

Changed ExprCons from a cdef class to a standard Python class and added type hints to the constructor parameters. This improves code readability and compatibility with Python tooling.

Type hints were added to methods in Term, Expr, PolynomialExpr, ConstExpr, ProdExpr, PowExpr, UnaryExpr, and ExprCons classes to improve code clarity and static analysis. Minor refactoring was performed for consistency in variable naming and method signatures.

Replaces the _to_unary_expr helper with a static method UnaryExpr.from_expr for creating unary expressions. Updates all relevant methods to use the new static method for consistency and improved encapsulation.

Changed the expected exception from NotImplementedError to TypeError when powering an expression with sqrt(2) in test_expr_op_expr. This reflects the actual exception raised by the code.

Changed quicksum and quickprod to use ConstExpr for initialization and updated parameter names and types for clarity. This improves performance and code readability by avoiding unnecessary intermediate data structures.

Refines the __add__ methods in Expr and SumExpr to better handle addition with SumExpr instances, ensuring correct merging of terms and consistent behavior when combining expressions.

Moved and unified the degree() method implementations for Term, Expr, PolynomialExpr, and FuncExpr classes. The degree calculation now consistently uses the degree() method of child elements, improving maintainability and correctness.

Corrects the logic for building node lists in Term and Expr classes to ensure proper handling of coefficients and child nodes. This improves the robustness of SCIP expression construction.

Updates the Expr.degree() method to return infinity when there are no children, instead of defaulting to zero. This ensures correct behavior for empty expressions.

Adds type checking for the expr argument and ensures at least one of lhs or rhs is provided in ExprCons. Moves validation logic from _normalize to __init__ for earlier error detection.

Initializes Expr children with {CONST: 0.0} by default instead of an empty dict. This ensures that expressions always have a constant term, which may help avoid issues with missing constants in further computations.

Updates the test to expect degree 0 for an empty Expr instead of infinity, aligning with the intended behavior of the Expr class.

Changed assertions in test_equation to expect lhs to be 1 instead of 0.0, reflecting updated behavior or requirements for equation evaluation.

Deleted the test_degree function from test_expr.py as it is no longer needed or used in the test suite.

Refines the __mul__ method in Expr to handle multiplication with zero and constant expressions more robustly. Also updates ConstExpr constructor to use float for default constant value.

Changed expected values in test_inequality to reflect updated behavior of GenExprs, asserting 1 instead of 0.0 for _lhs and _rhs.

This reverts commit 659fb2b.

Introduced a shared _remove_zero() method in SumExpr to eliminate zero-valued children, and updated normalization methods in SumExpr and PolynomialExpr to use this helper for improved code reuse and clarity.

Refactored __add__ in Expr to return 'other' when 'self' has no children, ensuring correct addition semantics and simplifying logic.

Changed the expected degree of an empty Expr from 0 to float('inf') in test_degree to reflect updated behavior.

Improves multiplication handling in Expr and ProdExpr classes. Expr now multiplies by a constant using dictionary comprehension, and ProdExpr prevents duplicate children and simplifies multiplication logic for constants.

Merged SumExpr functionality into Expr, simplifying sum expression logic and normalization. Updated PolynomialExpr to inherit directly from Expr. Adjusted Model class to handle Expr instead of SumExpr for sum expressions. This refactor streamlines expression management and reduces class complexity.

Moved Cython and C imports below standard library imports for better organization. Reformatted the addMatrixVar method signature and the addCons loop for improved readability and consistency.

Implements the __iadd__ method for PolynomialExpr, allowing in-place addition of polynomial expressions by updating child coefficients. Falls back to superclass behavior for non-polynomial operands.

Zeroto521 · 2025-11-26T13:28:11Z

Finish the base feature and function. And something needs to be done:

Copilot

Pull request overview

Copilot reviewed 14 out of 14 changed files in this pull request and generated 9 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2025-11-26T13:35:29Z

src/pyscipopt/expr.pxi

+            raise TypeError("expr must be an Expr instance")
+        if lhs is None and rhs is None:
+            raise ValueError(
+                "Ranged ExprCons (with both lhs and rhs) doesn't supported"


The error message is grammatically incorrect. It should be "Ranged ExprCons (with both lhs and rhs) isn't supported" or "Ranged ExprCons (with both lhs and rhs) is not supported" instead of "doesn't supported".

Suggested change

"Ranged ExprCons (with both lhs and rhs) doesn't supported"

"Ranged ExprCons (with both lhs and rhs) is not supported"

Copilot · 2025-11-26T13:35:30Z

src/pyscipopt/expr.pxi

+    def __next__(self) -> Union[Term, Expr]:
+        try:
+            return next(self.children)
+        except:


Using a bare except: clause is considered bad practice in Python. It should catch a specific exception type, such as except StopIteration:. A bare except: will catch all exceptions including SystemExit, KeyboardInterrupt, and others that should generally not be caught.

Suggested change

except:

except StopIteration:

Copilot · 2025-11-26T13:35:30Z

src/pyscipopt/scip.pxi

+        self = self.__add__(other)
+        return self


The __iadd__ implementation is incorrect. In Cython/Python, when self = self.__add__(other) is executed, the reassignment only affects the local variable self within the method, not the object itself. This means the in-place addition doesn't actually modify the Variable. While Variables are now immutable value objects (since they don't inherit from Expr), the __iadd__ should simply return the result of __add__ without the misleading assignment.

Suggested change

self = self.__add__(other)

return self

return self.__add__(other)

Copilot · 2025-11-26T13:35:31Z

src/pyscipopt/scip.pxi

    def _evaluate(self, term):
        self._checkStage("SCIPgetSolVal")
        result = 1
        cdef _VarArray wrapper
-        wrapper = _VarArray(term.vartuple)
-        for i in range(len(term.vartuple)):
+        wrapper = _VarArray(term.vars)
+        for i in range(len(term.vars)):
            result *= SCIPgetSolVal(self.scip, self.sol, wrapper.ptr[i])
        return result


The _evaluate method assumes that all children in expr.children are Term objects with a .vars attribute (line 1061). However, for non-polynomial expressions (like Expr({SinExpr(x): 1.0})), the children can be other Expr subclasses that don't have a .vars attribute, which will cause an AttributeError. The code should either check the type and handle non-Term children appropriately, or document that solution evaluation only works for polynomial expressions.

Copilot · 2025-11-26T13:35:31Z

tests/test_linexpr.py

    m, x, y, z = model
    expr = Expr()
-    assert expr.degree() == 0
+    assert expr.degree() == float("inf")


The degree of an empty Expr() should be 0 (representing a zero constant), not float("inf"). An empty expression has no variables and no terms, which is effectively a constant zero. The float("inf") degree is reserved for non-polynomial expressions like exp(), log(), etc.

Suggested change

assert expr.degree() == float("inf")

assert expr.degree() == 0

Copilot · 2025-11-26T13:35:31Z

src/pyscipopt/expr.pxi

+        return Expr(self._remove_zero())

+    def degree(self) -> float:
+        return max((i.degree() for i in self)) if self.children else float("inf")


The degree() method returns float("inf") for an empty expression, but this is inconsistent with the expected behavior. An empty Expr() should represent a zero constant and have degree 0, not infinity. The infinity degree should only apply to non-polynomial expressions (like those involving exp, log, sin, etc.). This affects the base implementation in line 244.

Suggested change

return max((i.degree() for i in self)) if self.children else float("inf")

return max((i.degree() for i in self)) if self.children else 0

Copilot · 2025-11-26T13:35:32Z

src/pyscipopt/expr.pxi

+                return ExprCons(self, lhs=other[CONST], rhs=other[CONST])
+            return (self - other).__eq__(0)
+        elif isinstance(other, MatrixExpr):
+            return other.__ge__(self)


The __eq__ method in line 206 calls other.__ge__(self) when other is a MatrixExpr, but this should call other.__eq__(self) to maintain the equality constraint semantics. Using __ge__ would incorrectly create a greater-than-or-equal constraint instead of an equality constraint.

Suggested change

return other.__ge__(self)

return other.__eq__(self)

Copilot · 2025-11-26T13:35:32Z

src/pyscipopt/expr.pxi

+
+
+class ExprCons:
+    """Constraints with a polynomial expressions and lower/upper bounds."""


The docstring says "Constraints with a polynomial expressions" but should say "Constraints with polynomial expressions" (without "a" before "polynomial"). Also note that ExprCons can now handle non-polynomial expressions too (like exp, log, etc.), so the docstring is inaccurate and should be updated to reflect that it handles general expressions.

Suggested change

"""Constraints with a polynomial expressions and lower/upper bounds."""

"""Constraints with general expressions and lower/upper bounds."""

Copilot · 2025-11-26T13:35:32Z

src/pyscipopt/scip.pxi

                    relevant_value = relevant_value[:-1] # removing %

-                if _is_number(relevant_value):
+                if isinstance(relevant_value, Number):


The check isinstance(relevant_value, Number) will always be False because relevant_value is a string extracted from the file (see line 12083 and 12117). Strings are never instances of Number. This appears to be a logic error introduced when replacing the old _is_number() helper function. The code should use a try/except block or recreate the _is_number() function to check if the string can be converted to a number.

Joao-Dionisio · 2025-11-26T22:27:34Z

Thanks for your work @Zeroto521 !! This is something that should be very very well tested before merging. We're also in the middle of a SCIP meeting, so I can bring this up with the guys and we can quickly take a high level look.

Zeroto521 added 20 commits October 28, 2025 10:19

Feature: merge Expr and GenExpr

58c34c0

Rename _unary to _to_unary_expr in expr.pxi

4cbc603

Remove Variable.create

188b3ef

Specify type of _lhs and _rhs as object in ExprCons

193ec1c

Changed the type of _lhs and _rhs attributes in the ExprCons class from unspecified to 'object' to clarify their expected type and improve type safety.

Merge branch 'master' into issue/1074

c70e305

lint codes

d8d63d2

Change Variable constructor to use __cinit__

a0086d2

Replaces the __init__ method with __cinit__ in the Variable class and updates the argument type to SCIP_VAR*.

MAINT: Support return solution

123f36e

Add return type annotations to _normalize methods

c961db5

Remove adding 0

0b01b08

MAINT: use class inner method to instead

28e6673

Replace 'terms' with 'children' in Expr usage

e4f48b5

Updated references from 'terms' to 'children' for Expr objects throughout Model methods to reflect changes in the Expr API. This ensures compatibility with the updated data structure and avoids errors when accessing expression terms.

lint codes

7d85fd0

Correct _evaluate cython syntax

f7159a0

Correct name: PowerExpr → PowExpr

69737c0

Merge branch 'master' into issue/1074

2b0fa85

Revert 188b3ef "Remove Variable.create"

0b7ee70

Refactor Expr to standard Python class

cc28588

Changed Expr from a Cython cdef class to a standard Python class for improved compatibility and maintainability. Removed cdef public dict children, as attribute is now managed in Python.

Simplify comparison

8719b91

Zeroto521 force-pushed the issue/1074 branch from c13b346 to 8719b91 Compare November 18, 2025 09:47

Zeroto521 added 2 commits November 18, 2025 17:50

Revert "MAINT: use class inner method to instead"

84886b3

This reverts commit 28e6673.

Change cdef classes to Python classes in expr.pxi

955a9e0

Converted SumExpr, ProdExpr, and PowExpr from cdef classes to regular Python classes for improved compatibility and maintainability.

Joao-Dionisio requested a review from Copilot November 18, 2025 10:11

Copilot started reviewing on behalf of Joao-Dionisio November 18, 2025 10:11 View session

Copilot finished reviewing on behalf of Joao-Dionisio November 18, 2025 10:12

Copilot AI reviewed Nov 18, 2025

View reviewed changes

src/pyscipopt/expr.pxi Outdated Show resolved Hide resolved

Zeroto521 and others added 3 commits November 18, 2025 18:35

lint codes

88aa0b4

Co-authored-by: Copilot <[email protected]>

Remove unused Expr class from scip.pxd

d4bf9b7

Deleted the definition of the Expr class, which was not used in the code. This helps clean up the codebase and improves maintainability.

Refactor ExprCons to Python class with type hints

b5f0435

Changed ExprCons from a cdef class to a standard Python class and added type hints to the constructor parameters. This improves code readability and compatibility with Python tooling.

Zeroto521 added 23 commits November 23, 2025 17:16

Move _to_unary_expr to UnaryExpr class inner

d74fbfa

Replaces the _to_unary_expr helper with a static method UnaryExpr.from_expr for creating unary expressions. Updates all relevant methods to use the new static method for consistency and improved encapsulation.

Update exception type in power operation test

882a1b1

Changed the expected exception from NotImplementedError to TypeError when powering an expression with sqrt(2) in test_expr_op_expr. This reflects the actual exception raised by the code.

Refactor quicksum and quickprod implementations

b971304

Changed quicksum and quickprod to use ConstExpr for initialization and updated parameter names and types for clarity. This improves performance and code readability by avoiding unnecessary intermediate data structures.

Improve addition logic for Expr classe

d56b5db

Refines the __add__ methods in Expr and SumExpr to better handle addition with SumExpr instances, ensuring correct merging of terms and consistent behavior when combining expressions.

Refactor degree calculation in expression classes

ca5aae2

Moved and unified the degree() method implementations for Term, Expr, PolynomialExpr, and FuncExpr classes. The degree calculation now consistently uses the degree() method of child elements, improving maintainability and correctness.

Fix node construction logic in expression classes

3a599a5

Corrects the logic for building node lists in Term and Expr classes to ensure proper handling of coefficients and child nodes. This improves the robustness of SCIP expression construction.

Fix degree calculation for empty expression children

0e7223b

Updates the Expr.degree() method to return infinity when there are no children, instead of defaulting to zero. This ensures correct behavior for empty expressions.

Improve ExprCons initialization and validation

86d60f7

Adds type checking for the expr argument and ensures at least one of lhs or rhs is provided in ExprCons. Moves validation logic from _normalize to __init__ for earlier error detection.

Set default CONST term in Expr children

659fb2b

Initializes Expr children with {CONST: 0.0} by default instead of an empty dict. This ensures that expressions always have a constant term, which may help avoid issues with missing constants in further computations.

Fix degree test for empty expression

36f46cc

Updates the test to expect degree 0 for an empty Expr instead of infinity, aligning with the intended behavior of the Expr class.

Update expected values in equation tests

726561c

Changed assertions in test_equation to expect lhs to be 1 instead of 0.0, reflecting updated behavior or requirements for equation evaluation.

Remove unused test_degree (it for GenExpr)

0edf545

Deleted the test_degree function from test_expr.py as it is no longer needed or used in the test suite.

Improve multiplication logic in Expr class

1b4719e

Refines the __mul__ method in Expr to handle multiplication with zero and constant expressions more robustly. Also updates ConstExpr constructor to use float for default constant value.

Update inequality test assertions in test_expr.py

1b481eb

Changed expected values in test_inequality to reflect updated behavior of GenExprs, asserting 1 instead of 0.0 for _lhs and _rhs.

Revert "Set default CONST term in Expr children"

cb9e8c2

This reverts commit 659fb2b.

Refactor zero-removal logic in SumExpr and PolynomialExpr

a679683

Introduced a shared _remove_zero() method in SumExpr to eliminate zero-valued children, and updated normalization methods in SumExpr and PolynomialExpr to use this helper for improved code reuse and clarity.

Fix addition behavior for Expr with no children

835feeb

Refactored __add__ in Expr to return 'other' when 'self' has no children, ensuring correct addition semantics and simplifying logic.

Update degree test for empty expression

1cf3a79

Changed the expected degree of an empty Expr from 0 to float('inf') in test_degree to reflect updated behavior.

Refactor Expr and ProdExpr multiplication logic

fe7027e

Improves multiplication handling in Expr and ProdExpr classes. Expr now multiplies by a constant using dictionary comprehension, and ProdExpr prevents duplicate children and simplifies multiplication logic for constants.

Reorder imports and reformat addMatrixVar and addCons loop

a0f3f57

Moved Cython and C imports below standard library imports for better organization. Reformatted the addMatrixVar method signature and the addCons loop for improved readability and consistency.

Speed up via avoid copying dict itself

8c9d155

Implements the __iadd__ method for PolynomialExpr, allowing in-place addition of polynomial expressions by updating child coefficients. Falls back to superclass behavior for non-polynomial operands.

Zeroto521 marked this pull request as ready for review November 26, 2025 13:26

Zeroto521 requested a review from Copilot November 26, 2025 13:28

Copilot started reviewing on behalf of Zeroto521 November 26, 2025 13:28 View session

Copilot finished reviewing on behalf of Zeroto521 November 26, 2025 13:33

Copilot AI reviewed Nov 26, 2025

View reviewed changes

	"Ranged ExprCons (with both lhs and rhs) doesn't supported"
	"Ranged ExprCons (with both lhs and rhs) is not supported"

	self = self.__add__(other)
	return self
	return self.__add__(other)

	assert expr.degree() == float("inf")
	assert expr.degree() == 0

	return max((i.degree() for i in self)) if self.children else float("inf")
	return max((i.degree() for i in self)) if self.children else 0



		class ExprCons:
		"""Constraints with a polynomial expressions and lower/upper bounds."""

	"""Constraints with a polynomial expressions and lower/upper bounds."""
	"""Constraints with general expressions and lower/upper bounds."""

Feature: Merge Expr and GenExpr #1114

Are you sure you want to change the base?

Feature: Merge Expr and GenExpr #1114

Uh oh!

Conversation

Zeroto521 commented Nov 18, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Zeroto521 commented Nov 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Nov 26, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 26, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 26, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 26, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 26, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 26, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 26, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 26, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 26, 2025

Choose a reason for hiding this comment

Uh oh!

Joao-Dionisio commented Nov 26, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Feature: Merge `Expr` and `GenExpr` #1114

Feature: Merge `Expr` and `GenExpr` #1114

Zeroto521 commented Nov 26, 2025 •

edited

Loading