Thank you all.
It turned out that the behavior is a little different depending on when I start the endpoint. If I start my endpoint during web app start up (ConfigureService), I will get null back from the injected IHttpContextAccessor. However, if I start the endpoint by the first http request, I will have a http context (from IHttpContextAccessor) and will always have access to it. I had mixed results in my test regarding if this context is distinct with different invocation, but I’m convinced that HttpContext isn’t reliable in message handler to pass ambient context. I will look into AsyncLocal<> for that.