Skip to content

Handle primitive collections as multiple parameters. #36157

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -163,10 +163,9 @@ public virtual TBuilder ExecutionStrategy(
/// </summary>
/// <remarks>
/// <para>
/// When a LINQ query contains a parameterized collection, by default EF Core parameterizes the entire collection as a single
/// SQL parameter, if possible. For example, on SQL Server, the LINQ query <c>Where(b => ids.Contains(b.Id)</c> is translated to
/// <c>WHERE [b].[Id] IN (SELECT [i].[value] FROM OPENJSON(@__ids_0) ...)</c>. While this helps with query plan caching, it can
/// produce worse query plans for certain query types.
/// When a LINQ query contains a parameterized collection, by default EF Core translates as a multiple SQL parameters,
/// if possible. For example, on SQL Server, the LINQ query <c>Where(b => ids.Contains(b.Id)</c> is translated to
/// <c>WHERE [b].[Id] IN (@ids1, @ids2, @ids3)</c>.
/// </para>
/// <para>
/// <see cref="TranslateParameterizedCollectionsToConstants" /> instructs EF to translate the collection to a set of constants:
Expand All @@ -176,37 +175,57 @@ public virtual TBuilder ExecutionStrategy(
/// <para>
/// Note that it's possible to cause EF to translate a specific collection in a specific query to constants by wrapping the
/// parameterized collection in <see cref="EF.Constant{T}" />: <c>Where(b => EF.Constant(ids).Contains(b.Id)</c>. This overrides
/// the default. Likewise, you can translate a specific collection in a specific query to a single parameter by wrapping the
/// parameterized collection in <see cref="EF.Parameter{T}(T)" />: <c>Where(b => EF.Parameter(ids).Contains(b.Id)</c>. This
/// overrides the <see cref="TranslateParameterizedCollectionsToConstants" /> setting.
/// the default.
/// </para>
/// </remarks>
public virtual TBuilder TranslateParameterizedCollectionsToConstants()
=> WithOption(e => (TExtension)e.WithParameterizedCollectionTranslationMode(ParameterizedCollectionTranslationMode.Constantize));

/// <summary>
/// Configures the context to translate parameterized collections to parameters.
/// Configures the context to translate parameterized collections to parameter.
/// </summary>
/// <remarks>
/// <para>
/// When a LINQ query contains a parameterized collection, by default EF Core parameterizes the entire collection as a single
/// SQL parameter, if possible. For example, on SQL Server, the LINQ query <c>Where(b => ids.Contains(b.Id)</c> is translated to
/// <c>WHERE [b].[Id] IN (SELECT [i].[value] FROM OPENJSON(@__ids_0) ...)</c>. While this helps with query plan caching, it can
/// produce worse query plans for certain query types.
/// When a LINQ query contains a parameterized collection, by default EF Core translates as a multiple SQL parameters,
/// if possible. For example, on SQL Server, the LINQ query <c>Where(b => ids.Contains(b.Id)</c> is translated to
/// <c>WHERE [b].[Id] IN (@ids1, @ids2, @ids3)</c>.
/// </para>
/// <para>
/// <see cref="TranslateParameterizedCollectionsToParameters" /> explicitly instructs EF to perform the default translation
/// of parameterized collections, which is translating them to parameters.
/// <see cref="TranslateParameterizedCollectionsToParameters" /> instructs EF to translate the collection to a set of constants:
/// <c>WHERE [b].[Id] IN (SELECT [i].[value] FROM OPENJSON(@ids) ...)</c>.
/// </para>
/// <para>
/// Note that it's possible to cause EF to translate a specific collection in a specific query to constants by wrapping the
/// parameterized collection in <see cref="EF.Constant{T}" />: <c>Where(b => EF.Constant(ids).Contains(b.Id)</c>. This overrides
/// Note that it's possible to cause EF to translate a specific collection in a specific query to parameter by wrapping the
/// parameterized collection in <see cref="EF.Parameter{T}" />: <c>Where(b => EF.Parameter(ids).Contains(b.Id)</c>. This overrides
/// the default.
/// </para>
/// </remarks>
public virtual TBuilder TranslateParameterizedCollectionsToParameters()
=> WithOption(e => (TExtension)e.WithParameterizedCollectionTranslationMode(ParameterizedCollectionTranslationMode.Parameterize));

/// <summary>
/// Configures the context to translate parameterized collections to expanded parameters.
/// </summary>
/// <remarks>
/// <para>
/// When a LINQ query contains a parameterized collection, by default EF Core translates as a multiple SQL parameters,
/// if possible. For example, on SQL Server, the LINQ query <c>Where(b => ids.Contains(b.Id)</c> is translated to
/// <c>WHERE [b].[Id] IN (@ids1, @ids2, @ids3)</c>.
/// </para>
/// <para>
/// <see cref="TranslateParameterizedCollectionsToExpandedParameters" /> instructs EF to translate the collection to a set of parameters:
/// <c>WHERE [b].[Id] IN (@ids1, @ids2, @ids3)</c>.
/// </para>
/// </remarks>
//TODO: When appropriate EF method is implemented, mention it here.
// <para>
// Note that it's possible to cause EF to translate a specific collection in a specific query to expanded parameters by wrapping the
// parameterized collection in <see cref="EF.???{T}" />: <c>Where(b => EF.Parameter(ids).???(b.Id)</c>. This overrides
// the default.
// </para>
public virtual TBuilder TranslateParameterizedCollectionsToExpandedParameters()
=> WithOption(e => (TExtension)e.WithParameterizedCollectionTranslationMode(ParameterizedCollectionTranslationMode.ParameterizeExpanded));

/// <summary>
/// Sets an option by cloning the extension used to store the settings. This ensures the builder
/// does not modify options that are already in use elsewhere.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,4 +26,12 @@ public enum ParameterizedCollectionTranslationMode
/// doing so can result in application failures when updating to a new Entity Framework Core release.
/// </summary>
Parameterize,

/// <summary>
/// This is an internal API that supports the Entity Framework Core infrastructure and not subject to
/// the same compatibility standards as public APIs. It may be changed or removed without notice in
/// any release. You should only use it directly in your code with extreme caution and knowing that
/// doing so can result in application failures when updating to a new Entity Framework Core release.
/// </summary>
ParameterizeExpanded,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The name is obviously to discuss.

}
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@

using System.Collections.Concurrent;
using System.Runtime.CompilerServices;
using Microsoft.EntityFrameworkCore.Internal;
using Microsoft.Extensions.Caching.Memory;

namespace Microsoft.EntityFrameworkCore.Query.Internal;
Expand Down Expand Up @@ -34,13 +35,14 @@ public RelationalCommandCache(
IQuerySqlGeneratorFactory querySqlGeneratorFactory,
IRelationalParameterBasedSqlProcessorFactory relationalParameterBasedSqlProcessorFactory,
Expression queryExpression,
bool useRelationalNulls)
bool useRelationalNulls,
ParameterizedCollectionTranslationMode? parameterizedCollectionTranslationMode)
{
_memoryCache = memoryCache;
_querySqlGeneratorFactory = querySqlGeneratorFactory;
_queryExpression = queryExpression;
_relationalParameterBasedSqlProcessor = relationalParameterBasedSqlProcessorFactory.Create(
new RelationalParameterBasedSqlProcessorParameters(useRelationalNulls));
new RelationalParameterBasedSqlProcessorParameters(useRelationalNulls, parameterizedCollectionTranslationMode));
}

/// <summary>
Expand All @@ -49,7 +51,7 @@ public RelationalCommandCache(
/// any release. You should only use it directly in your code with extreme caution and knowing that
/// doing so can result in application failures when updating to a new Entity Framework Core release.
/// </summary>
public virtual IRelationalCommandTemplate GetRelationalCommandTemplate(IReadOnlyDictionary<string, object?> parameters)
public virtual IRelationalCommandTemplate GetRelationalCommandTemplate(Dictionary<string, object?> parameters)
{
var cacheKey = new CommandCacheKey(_queryExpression, parameters);

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,4 +9,4 @@ namespace Microsoft.EntityFrameworkCore.Query.Internal;
/// any release. You should only use it directly in your code with extreme caution and knowing that
/// doing so can result in application failures when updating to a new Entity Framework Core release.
/// </summary>
public delegate IRelationalCommandTemplate RelationalCommandResolver(IReadOnlyDictionary<string, object?> parameters);
public delegate IRelationalCommandTemplate RelationalCommandResolver(Dictionary<string, object?> parameters);
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ public RelationalParameterBasedSqlProcessor(
/// <returns>An optimized query expression.</returns>
public virtual Expression Optimize(
Expression queryExpression,
IReadOnlyDictionary<string, object?> parametersValues,
Dictionary<string, object?> parametersValues,
out bool canCache)
{
canCache = true;
Expand All @@ -72,7 +72,7 @@ public virtual Expression Optimize(
/// <returns>A processed query expression.</returns>
protected virtual Expression ProcessSqlNullability(
Expression queryExpression,
IReadOnlyDictionary<string, object?> parametersValues,
Dictionary<string, object?> parametersValues,
out bool canCache)
=> new SqlNullabilityProcessor(Dependencies, Parameters).Process(queryExpression, parametersValues, out canCache);

Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
// Licensed to the .NET Foundation under one or more agreements.
// The .NET Foundation licenses this file to you under the MIT license.

using Microsoft.EntityFrameworkCore.Internal;

namespace Microsoft.EntityFrameworkCore.Query;

/// <summary>
Expand All @@ -13,11 +15,20 @@ public sealed record RelationalParameterBasedSqlProcessorParameters
/// </summary>
public bool UseRelationalNulls { get; init; }

/// <summary>
/// A value indicating what translation mode should be used.
/// </summary>
public ParameterizedCollectionTranslationMode? ParameterizedCollectionTranslationMode { get; init; }

/// <summary>
/// Creates a new instance of <see cref="RelationalParameterBasedSqlProcessorParameters" />.
/// </summary>
/// <param name="useRelationalNulls">A value indicating if relational nulls should be used.</param>
/// <param name="parameterizedCollectionTranslationMode">A value indicating what translation mode should be used.</param>
[EntityFrameworkInternal]
public RelationalParameterBasedSqlProcessorParameters(bool useRelationalNulls)
=> UseRelationalNulls = useRelationalNulls;
public RelationalParameterBasedSqlProcessorParameters(bool useRelationalNulls, ParameterizedCollectionTranslationMode? parameterizedCollectionTranslationMode)
{
UseRelationalNulls = useRelationalNulls;
ParameterizedCollectionTranslationMode = parameterizedCollectionTranslationMode;
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -295,10 +295,10 @@ JsonScalarExpression jsonScalar

var primitiveCollectionsBehavior = RelationalOptionsExtension.Extract(QueryCompilationContext.ContextOptions)
.ParameterizedCollectionTranslationMode;

var tableAlias = _sqlAliasManager.GenerateTableAlias(sqlParameterExpression.Name.TrimStart('_'));

if (queryParameter.ShouldBeConstantized
|| (primitiveCollectionsBehavior == ParameterizedCollectionTranslationMode.Constantize
|| (primitiveCollectionsBehavior is ParameterizedCollectionTranslationMode.Constantize
&& !queryParameter.ShouldNotBeConstantized))
{
var valuesExpression = new ValuesExpression(
Expand All @@ -313,6 +313,21 @@ JsonScalarExpression jsonScalar
sqlParameterExpression.IsNullable);
}

if ((primitiveCollectionsBehavior is null or ParameterizedCollectionTranslationMode.ParameterizeExpanded)
&& !queryParameter.ShouldNotBeConstantized)
{
var valuesExpression = new ValuesExpression(
tableAlias,
sqlParameterExpression,
[ValuesOrderingColumnName, ValuesValueColumnName]);
return CreateShapedQueryExpressionForValuesExpression(
valuesExpression,
tableAlias,
parameterQueryRootExpression.ElementType,
sqlParameterExpression.TypeMapping,
sqlParameterExpression.IsNullable);
}

return TranslatePrimitiveCollection(sqlParameterExpression, property: null, tableAlias);
}

Expand Down Expand Up @@ -569,16 +584,23 @@ protected override ShapedQueryExpression TranslateConcat(ShapedQueryExpression s
return TranslateAny(source, anyLambda);
}

// Pattern-match Contains over ValuesExpression, translating to simplified 'item IN (1, 2, 3)' with constant elements
var primitiveCollectionsBehavior = RelationalOptionsExtension.Extract(QueryCompilationContext.ContextOptions)
.ParameterizedCollectionTranslationMode;
// Pattern-match Contains over ValuesExpression, translating to simplified 'item IN (1, 2, 3)' with constant elements.
if (TryExtractBareInlineCollectionValues(source, out var values, out var valuesParameter))
{
var inExpression = (values, valuesParameter) switch
if (values is not null)
{
(not null, null) => _sqlExpressionFactory.In(translatedItem, values),
(null, not null) => _sqlExpressionFactory.In(translatedItem, valuesParameter),
_ => throw new UnreachableException(),
};
return source.Update(new SelectExpression(inExpression, _sqlAliasManager), source.ShaperExpression);
var inExpression = _sqlExpressionFactory.In(translatedItem, values);
return source.Update(new SelectExpression(inExpression, _sqlAliasManager), source.ShaperExpression);
}
if (valuesParameter is not null
// Expanding parameters will happen in 2nd stage of query pipeline.
&& primitiveCollectionsBehavior is not (null or ParameterizedCollectionTranslationMode.ParameterizeExpanded))
{
var inExpression = _sqlExpressionFactory.In(translatedItem, valuesParameter);
return source.Update(new SelectExpression(inExpression, _sqlAliasManager), source.ShaperExpression);
}
}

// Translate to IN with a subquery.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ public partial class RelationalShapedQueryCompilingExpressionVisitor : ShapedQue
private readonly bool _threadSafetyChecksEnabled;
private readonly bool _detailedErrorsEnabled;
private readonly bool _useRelationalNulls;
private readonly ParameterizedCollectionTranslationMode? _parameterizedCollectionTranslationMode;
private readonly bool _isPrecompiling;

private readonly RelationalParameterBasedSqlProcessor _relationalParameterBasedSqlProcessor;
Expand Down Expand Up @@ -54,14 +55,15 @@ public RelationalShapedQueryCompilingExpressionVisitor(

_relationalParameterBasedSqlProcessor =
relationalDependencies.RelationalParameterBasedSqlProcessorFactory.Create(
new RelationalParameterBasedSqlProcessorParameters(_useRelationalNulls));
new RelationalParameterBasedSqlProcessorParameters(_useRelationalNulls, _parameterizedCollectionTranslationMode));
_querySqlGeneratorFactory = relationalDependencies.QuerySqlGeneratorFactory;

_contextType = queryCompilationContext.ContextType;
_tags = queryCompilationContext.Tags;
_threadSafetyChecksEnabled = dependencies.CoreSingletonOptions.AreThreadSafetyChecksEnabled;
_detailedErrorsEnabled = dependencies.CoreSingletonOptions.AreDetailedErrorsEnabled;
_useRelationalNulls = RelationalOptionsExtension.Extract(queryCompilationContext.ContextOptions).UseRelationalNulls;
_parameterizedCollectionTranslationMode = RelationalOptionsExtension.Extract(queryCompilationContext.ContextOptions).ParameterizedCollectionTranslationMode;
_isPrecompiling = queryCompilationContext.IsPrecompiling;
}

Expand Down Expand Up @@ -497,15 +499,16 @@ private Expression CreateRelationalCommandResolverExpression(Expression queryExp
RelationalDependencies.QuerySqlGeneratorFactory,
RelationalDependencies.RelationalParameterBasedSqlProcessorFactory,
queryExpression,
_useRelationalNulls);
_useRelationalNulls,
_parameterizedCollectionTranslationMode);

var commandLiftableConstant = RelationalDependencies.RelationalLiftableConstantFactory.CreateLiftableConstant(
relationalCommandCache,
GenerateRelationalCommandCacheExpression(),
"relationalCommandCache",
typeof(RelationalCommandCache));

var parametersParameter = Parameter(typeof(IReadOnlyDictionary<string, object?>), "parameters");
var parametersParameter = Parameter(typeof(Dictionary<string, object?>), "parameters");

return Lambda<RelationalCommandResolver>(
Call(
Expand Down Expand Up @@ -542,7 +545,7 @@ bool TryGeneratePregeneratedCommandResolver(
return false;
}

var parameterDictionaryParameter = Parameter(typeof(IReadOnlyDictionary<string, object?>), "parameters");
var parameterDictionaryParameter = Parameter(typeof(Dictionary<string, object?>), "parameters");
var resultParameter = Parameter(typeof(IRelationalCommandTemplate), "result");
Expression resolverBody;
bool canCache;
Expand Down Expand Up @@ -657,7 +660,7 @@ static object GenerateNonNullParameterValue(Type type)
}
}

Expression GenerateRelationalCommandExpression(IReadOnlyDictionary<string, object?> parameters, out bool canCache)
Expression GenerateRelationalCommandExpression(Dictionary<string, object?> parameters, out bool canCache)
{
var queryExpression = _relationalParameterBasedSqlProcessor.Optimize(select, parameters, out canCache);
if (!canCache)
Expand Down Expand Up @@ -747,7 +750,8 @@ Expression<Func<RelationalMaterializerLiftableConstantContext, object>> Generate
MakeMemberAccess(contextParameter, _relationalDependenciesProperty),
_relationalDependenciesRelationalParameterBasedSqlProcessorFactoryProperty),
Constant(queryExpression),
Constant(_useRelationalNulls)),
Constant(_useRelationalNulls),
Constant(_parameterizedCollectionTranslationMode, typeof(ParameterizedCollectionTranslationMode?))),
contextParameter);
}
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -167,7 +167,7 @@ public override Expression Quote()
Constant(Alias, typeof(string)),
RowValues is not null
? NewArrayInit(typeof(RowValueExpression), RowValues.Select(rv => rv.Quote()))
: Constant(null, typeof(RowValueExpression)),
: Constant(null, typeof(IReadOnlyList<RowValueExpression>)),
RelationalExpressionQuotingUtilities.QuoteOrNull(ValuesParameter),
NewArrayInit(typeof(string), ColumnNames.Select(Constant)),
RelationalExpressionQuotingUtilities.QuoteAnnotations(Annotations));
Expand Down
Loading