Azure KeyVault: Azure.Identity.CredentialUnavailableException: DefaultAzureCredential failed to retrieve a token from the included credentials

I am trying to connect my ASP.NET Core application that is targeting .NET framework with Azure Keyvault. On a new Azure VM that supports Identity everything works fine, but this application is hosted on a classic Azure VM that does not support Identity. I made the system environment variable AzureServiceAuthConnectionString which several other .NET framework applications with Azure Keyvault are already using and they are working perfectly.

Looking at my stdout logs I get the following exception every time:

Azure.Identity.CredentialUnavailableException: DefaultAzureCredential failed to retrieve a token from the included credentials EnvironmentCredential authentication unavailable. Environment variables are not fully configured ManagedIdentityCredential authentication unavailable, the requested identity has not been assigned to this resource.

I use the following code in the startup:

public static IWebHostBuilder CreateWebHostBuilder(string[] args) =>
            WebHost.CreateDefaultBuilder(args)               
       .UseApplicationInsights(ConfigurationManager.AppSettings["applicationInsightsInstrumentationKey"])
                .ConfigureKestrel(options => options.AddServerHeader = false)
                .UseIISIntegration()
                .ConfigureAppConfiguration((context, config) =>
                {
                    var vaultName = ConfigurationManager.AppSettings["VaultName"];
                    if (!string.IsNullOrEmpty(vaultName))
                    {
                        var azureServiceTokenProvider = new AzureServiceTokenProvider();
                        var keyVaultClient = new KeyVaultClient(
                            new KeyVaultClient.AuthenticationCallback(
                                azureServiceTokenProvider.KeyVaultTokenCallback));

                        config.AddAzureKeyVault(
                            $"https://{vaultName}.vault.azure.net/",
                            keyVaultClient,
                            new DefaultKeyVaultSecretManager());
                    }
                })
                .UseStartup<Startup>();

And in the web.config the following items :

<configSections>
    <section name="configBuilders" type="System.Configuration.ConfigurationBuildersSection, System.Configuration, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" restartOnExternalChanges="false" requirePermission="false"/>
</configSections>
<configBuilders>
    <builders>
        <add name="AzureKeyVault" vaultName="<#= this.VaultName #>" type="Microsoft.Configuration.ConfigurationBuilders.AzureKeyVaultConfigBuilder, Microsoft.Configuration.ConfigurationBuilders.Azure, Version=2.0.0.0, Culture=neutral" vaultUri="https://<#= this.VaultName #>.vault.azure.net" />
    </builders>
</configBuilders>
<connectionStrings configBuilders="AzureKeyVault">
      <add name="ConnectionString" connectionString="" providerName="System.Data.SqlClient"/>
</connectionStrings>