Amazon OpenSearch的设置Serilog

Question

我正在尝试设置Serilog，以便将日志从ASP.NET Core WebAPI发送到Amazon OpenSearch的本地实例。我在控制台上看到了日志，但是OpenSearch中没有显示任何内容。

已安装的第三方库：

• Serilog.aspnetcore（6.0.0-dev-00265）
• Serilog.enrichers.enrichers.environment（2.2.1-dev-00787）
• Serilog.sink.sinks.sinks.elasticsearch（9.0.0.0.0.0-beta7）通过开发Docker Compose（9.0.0.0-Beta）

运行（没有安全插件）：

https://opensearch.org/docs/2.0/opensearch/install/docker/#sample-docker-docker-docker-docker-file-for-for-development

program.cs

var logger = new LoggerConfiguration()
      .WriteTo.Console()
      .WriteTo.Elasticsearch(new ElasticsearchSinkOptions(new Uri("http://localhost:9200"))
      {
          AutoRegisterTemplate = true,
          MinimumLogEventLevel = LogEventLevel.Information,  
          FailureCallback = FailureCallback,
          EmitEventFailure = EmitEventFailureHandling.RaiseCallback | EmitEventFailureHandling.ThrowException
      })
      .CreateLogger();

builder.Logging.ClearProviders();
builder.Logging.AddSerilog(logger);

controller类：

_logger.LogWarning("Example warning");
_logger.LogError("Example error");

failurecallback是空的。 OpenSearch Console不会显示任何问题。

可能怎么了？

Answer 1

我已经尝试了您的设置，这里有一些结果（仅使用稳定版本的软件注意）：.

• net core v6.0（不是beta）
• serilog.sinks.sinks.elasticsearch v 8.4.1
• serilog.aspnetcore 5.0.0

docker-compose二手：

version: '3'
services:
  opensearch-node1:
    image: opensearchproject/opensearch:2.0.1
    container_name: opensearch-node1
    environment:
      - cluster.name=opensearch-cluster
      - node.name=opensearch-node1
      - bootstrap.memory_lock=true # along with the memlock settings below, disables swapping
      - "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
      - "DISABLE_INSTALL_DEMO_CONFIG=true" # disables execution of install_demo_configuration.sh bundled with security plugin, which installs demo certificates and security configurations to OpenSearch
      - "DISABLE_SECURITY_PLUGIN=true" # disables security plugin entirely in OpenSearch by setting plugins.security.disabled: true in opensearch.yml
      - "discovery.type=single-node" # disables bootstrap checks that are enabled when network.host is set to a non-loopback address
    ulimits:
      memlock:
        soft: -1
        hard: -1
      nofile:
        soft: 65536 # maximum number of open files for the OpenSearch user, set to at least 65536 on modern systems
        hard: 65536
    volumes:
      - opensearch-data1:/usr/share/opensearch/data
    ports:
      - 9200:9200
      - 9600:9600 # required for Performance Analyzer
    networks:
      - opensearch-net

  opensearch-dashboards:
    image: opensearchproject/opensearch-dashboards:2.0.1
    container_name: opensearch-dashboards
    ports:
      - 5601:5601
    expose:
      - "5601"
    environment:
      - 'OPENSEARCH_HOSTS=["http://opensearch-node1:9200"]'
      - "DISABLE_SECURITY_DASHBOARDS_PLUGIN=true" # disables security dashboards plugin in OpenSearch Dashboards
    networks:
      - opensearch-net

volumes:
  opensearch-data1:

networks:
  opensearch-net:

program.cs

using Serilog;
using Serilog.Events;
using Serilog.Sinks.Elasticsearch;

var builder = WebApplication.CreateBuilder(args);

// Add services to the container.
builder.Services.AddRazorPages();

builder.Logging.ClearProviders();

Serilog.Debugging.SelfLog.Enable(msg => Console.WriteLine(msg));

ServicePointManager.ServerCertificateValidationCallback =
    (source, certificate, chain, sslPolicyErrors) => true;

var logger = new LoggerConfiguration()
    .WriteTo.Console()
    .WriteTo.Elasticsearch(new ElasticsearchSinkOptions(new Uri("http://localhost:9200"))
    {
        AutoRegisterTemplate = true,
        MinimumLogEventLevel = LogEventLevel.Information,
        FailureCallback = e => Console.WriteLine("Unable to submit event " + e.MessageTemplate),
        EmitEventFailure = EmitEventFailureHandling.RaiseCallback | EmitEventFailureHandling.ThrowException,
        TypeName =  "_doc",
        InlineFields = false

    })
    .CreateLogger();

builder.Logging.ClearProviders();
builder.Logging.AddSerilog(logger);

    

var app = builder.Build();

// Configure the HTTP request pipeline.
if (!app.Environment.IsDevelopment())
{
    app.UseExceptionHandler("/Error");
    // The default HSTS value is 30 days. You may want to change this for production scenarios, see https://aka.ms/aspnetcore-hsts.
    app.UseHsts();
}


app.UseHttpsRedirection();
app.UseStaticFiles();

app.UseRouting();

app.UseAuthorization();

app.MapRazorPages();

app.Run();

，很少有帮助解决故障排除

1. serilog.debugging.selflog.enable（msg =＆gt; console.writeline（msg））; - 将显示Serilog的实际错误（很可能您会看到SSL）

2. service> servicepointpointManager中的实际错误。 serverCertificateValidationCallback =（源，证书，链，SSLPOLICYERRORS）=＆gt; true; - 现在停止任何SSL问题（您可以稍后修复它们）

3. 我捕获的下一件事是由Serilog生成的[_type]字段的问题，并且不被Elastic＆GT接受； v8.2，这很可能会因为您的缓冲区保留旧记录而发生。

4. 采用最新的beta版本用于使用typeName =“ _ doc”时，AWS OpenSearch 2.0.1有另一个错误，其中``compatibily.override_main_response_version = true''设置（请参阅此处的详细信息）
   https://github.com/opensearch-project/opensearch-project/openseact/openseact/pull/pull/pull/3530 - 基本上，我建议回滚AWS opensearch to V2。

之后，希望它应该起作用:)

Answer 2

对我来说，问题是我没有为indexformat属性提供值（在elasticsearchSinkoptions对象中）。相反，我将其放在端点上，就像您通过REST插入数据时应该做的那样。总而言之，以下代码为我解决了问题：

var jsonFormatter = new CompactJsonFormatter();

var loggerConfig = new LoggerConfiguration()
  .Enrich.FromLogContext()
  .WriteTo.Map("Name", "**error**", (name, writeTo) =>
  {
    var currentYear = DateTime.Today.Year;
    var currentWeek = calendar.GetWeekOfYear(DateTime.Now, 
                                CalendarWeekRule.FirstDay, 
                                DayOfWeek.Monday);
    writeTo.Elasticsearch(new ElasticsearchSinkOptions(new Uri("<opensearch endpoint>"))
    {
      CustomFormatter = jsonFormatter,
      TypeName = "_doc",
      IndexFormat = $"my-index-{currentYear}-{currentWeek}",
      MinimumLogEventLevel = LogEventLevel.Information,
      EmitEventFailure = EmitEventFailureHandling.RaiseCallback | 
                          EmitEventFailureHandling.ThrowException,
      FailureCallback = e =>
        Console.WriteLine(
          "An error occured in Serilog ElasticSearch sink: " +
          $"{e.Exception.Message} | {e.Exception.InnerException?.Message}")
    });
  });
Log.Logger = loggerConfig.CreateLogger();

当然，您还需要正确设置OpenSearch，以便它可以自动将策略自动到您的索引等。