Prevent direct access to files in a directory

I have a folder in my website.  Users will be able to upload files into this folder.  I want to be able to prevent anonymous users from downloading the files in this folder.

I am using the new ASP Identity for authentication, and it works perfectly fine for preventing access to Controllers and ASPX files - but not for files which are not ".NET files".

As an example, in this folder that I have named "test", I have placed a file called "file.html" and another file called "file.aspx".

I have also placed a web.config file in this folder and set it to deny access to anonymous users:

<?xml version="1.0"?>
    <configuration>
        <system.web>
            <authorization>
                <deny users="?" />
            </authorization>
        </system.web>
</configuration>

If I try to go to "localhost/test/file.aspx" it works correctly --- which is to redirect me to my login page.

If I try to go to "localhost/test/file.html" it does not work -- and allows the file to be viewed whether or not I log in.

I have read articles about writing custom HTTP handlers to help with this problem, and I have tried to do that also, but I cannot get the web server to execute the handler.  I am using Visual Studio 2013.  Does anyone have any additional suggestions that I could try?