首页 新闻 赞助 找找看

升级至 .NET 7 之后 System.Text.Json 在序列化大文本时会出现 OutOfMemoryException

0
悬赏园豆:30 [待解决问题]

在升级至 .NET 7 之后,已经在2个项目中遇到 System.Text.Json 在序列化大文本时会出现 OutOfMemoryException,序列化的对象中保存了博文内容,当博文内容字符数过多时就容易引发这个异常

以下是异常日志

System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
   at System.GC.AllocateNewArray(IntPtr typeHandle, Int32 length, GC_ALLOC_FLAGS flags)
   at System.GC.<AllocateUninitializedArray>g__AllocateNewUninitializedArray|66_0[T](Int32 length, Boolean pinned)
   at System.Buffers.TlsOverPerCoreLockedStacksArrayPool`1.Rent(Int32 minimumLength)
   at System.Text.Json.Utf8JsonWriter.WriteStringEscapeValue(ReadOnlySpan`1 value, Int32 firstEscapeIndexVal)
   at System.Text.Json.Serialization.JsonConverter`1.TryWrite(Utf8JsonWriter writer, T& value, JsonSerializerOptions options, WriteStack& state)
   at System.Text.Json.Serialization.Metadata.JsonPropertyInfo`1.GetMemberAndWriteJson(Object obj, WriteStack& state, Utf8JsonWriter writer)
   at System.Text.Json.Serialization.Converters.ObjectDefaultConverter`1.OnTryWrite(Utf8JsonWriter writer, T value, JsonSerializerOptions options, WriteStack& state)
   at System.Text.Json.Serialization.JsonConverter`1.TryWrite(Utf8JsonWriter writer, T& value, JsonSerializerOptions options, WriteStack& state)
   at System.Text.Json.Serialization.Converters.ListOfTConverter`2.OnWriteResume(Utf8JsonWriter writer, TCollection value, JsonSerializerOptions options, WriteStack& state)
   at System.Text.Json.Serialization.JsonCollectionConverter`2.OnTryWrite(Utf8JsonWriter writer, TCollection value, JsonSerializerOptions options, WriteStack& state)
   at System.Text.Json.Serialization.JsonConverter`1.TryWrite(Utf8JsonWriter writer, T& value, JsonSerializerOptions options, WriteStack& state)
   at System.Text.Json.Serialization.JsonConverter`1.WriteCore(Utf8JsonWriter writer, T& value, JsonSerializerOptions options, WriteStack& state)
   at System.Text.Json.Serialization.JsonConverter`1.WriteCoreAsObject(Utf8JsonWriter writer, Object value, JsonSerializerOptions options, WriteStack& state)
   at System.Text.Json.JsonSerializer.WriteCore[TValue](Utf8JsonWriter writer, TValue& value, JsonTypeInfo jsonTypeInfo, WriteStack& state)
   at System.Text.Json.JsonSerializer.WriteStreamAsync[TValue](Stream utf8Json, TValue value, JsonTypeInfo jsonTypeInfo, CancellationToken cancellationToken)
   at System.Text.Json.JsonSerializer.WriteStreamAsync[TValue](Stream utf8Json, TValue value, JsonTypeInfo jsonTypeInfo, CancellationToken cancellationToken)
   at System.Text.Json.JsonSerializer.WriteStreamAsync[TValue](Stream utf8Json, TValue value, JsonTypeInfo jsonTypeInfo, CancellationToken cancellationToken)
Microsoft.AspNetCore.Mvc.Formatters.SystemTextJsonOutputFormatter.WriteResponseBodyAsync(OutputFormatterWriteContext context, Encoding selectedEncoding)
问题补充:

从 System.Text.Json.Utf8JsonWriter 对应的实现代码 WriteStringEscapeValue 看,异常是在下面的代码中执行 ArrayPool<char>.Shared.Rent 时抛出的

Span<char> escapedValue = length <= JsonConstants.StackallocCharThreshold ?
    stackalloc char[JsonConstants.StackallocCharThreshold] :
    (valueArray = ArrayPool<char>.Shared.Rent(length));

ASP.NET Core 中对应的源码 SystemTextJsonOutputFormatter.cs#L82

await JsonSerializer.SerializeAsync(responseStream, context.Object, objectType, SerializerOptions, httpContext.RequestAborted);
dudu的主页 dudu | 高人七级 | 园豆:31576
提问于:2022-11-12 16:49

github 上的相关 issue https://github.com/dotnet/runtime/issues/66102

dudu 1年前

System.Text.Json.Utf8JsonWriter 对应的实现代码 WriteStringEscapeValue

dudu 1年前

github 上的相关 issue Multiple 'System.OutOfMemoryException' errors in .NET 7

dudu 1年前
< >
分享
所有回答(2)
0

GC提示内存不足,或许可以尝试调整垃圾回收器的设置;
https://learn.microsoft.com/zh-cn/dotnet/core/runtime-config/garbage-collector

diudiu1 | 园豆:1031 (小虾三级) | 2022-11-13 21:00
0

是 .NET 7 GC 的问题引起的

in .NET 7 we have enabled new Regions functionality within the GC. Here are the details: #43844. Since this was a foundational change, we also shipped a separate GC which keeps the previous "Segments" functionality -- in case there are some issues like this one.

1月会发布更新,目前的临时解决方法是添加环境变量 COMPlus_GCName=libclrgc.so(linux) 或者 COMPlus_GCName=clrgc.dll(windows)

详见 https://github.com/dotnet/runtime/issues/78959

dudu | 园豆:31576 (高人七级) | 2022-12-29 10:44

.NET 7.0.3 今天发布了,这个问题在 Fix spurious OOMs after upgrading to 7 中修复了

支持(0) 反对(0) dudu | 园豆:31576 (高人七级) | 2023-02-15 07:02

.NET 7.0.3 没有彻底解决问题,还是会出现 OutOfMemoryException

支持(0) 反对(0) dudu | 园豆:31576 (高人七级) | 2023-03-01 10:28
清除回答草稿
   您需要登录以后才能回答,未注册用户请先注册