Caching and upper case issue with BIND 9.9.7-P3

[cypher Nix]

On Wednesday, September 23, 2015 at 11:18:59 AM UTC-4, cypher Nix wrote:
> After upgrading BIND from BIND 9.9.7-P2 to BIND 9.9.7-P3 on about a dozen
> authoritative + recursive servers, we noticed a strange caching issue on one of the servers.
> 
> The server is authoritative for our main domain (let's assume example.com).
> There are multiple subdomains under example.com that have been NS delegated to
> other servers. Whenever the DNS server would respond to a recursive "A" record query from its
> cache, the "Answers" part of the request would always be in upper case - such
> as foo.bar.EXAMPLE.COM.
> More details below
> I noted the following behavior using a packet capture
> 
> -A client requests for "foo.bar.example.com" "A" record.
> 
> -Our server then does a lookup against bar.example.com authoritative server for "A" record "foo.bar.exampe.com" 
> 
> -Our server gets a response from bar.example.com authoritative server:
>     ; ANSWER SECTION:
>     foo.bar.example.com. 10 IN    A       192.168.0.1
> 
> -Our server responds to the clients request for foo.bar.example.com and stores
> the response for 10 seconds (the TTL of the record). At this point the
> answer section is still in all lower case - the clients gets the following:
>     ; ANSWER SECTION:
>     foo.bar.example.com. 10 IN    A       192.168.0.1
> 
> -The next time the client queries for foo.bar.exmaple.com, our server responds
> from the cache and changes the case from example.com to EXAMPLE.COM. It
> continues to serve EXAMPLE.COM in upper case as part of the answer while the TTL is still valid.
>     ; ANSWER SECTION:
>     foo.bar.EXAMPLE.COM. 9 IN    A       192.168.0.1
> 
> -This behavior was observed for "A" record responses for queries under any
> subdomain of example.com.  The case was only change to upper case on the
> answer section. Example.com also appeared under the question, authority, and
> additional sections but only in the answer section was the case changed.
> 
> We eventually restarted BIND and the issue went
> away. After restarting BIND all responses served from cache are now lower case, as expected.
> 
> Has anyone seen this behavior before ? Is this a bug ?
> This caused issues for certain applications on our network that did string
> comparison and expected the answer section to be in lower case.

Thank you all for your help. I was able to reproduce this behavior in the lab using older versions of BIND (9.9.7 P2 and 9.9.6 P1). I setup two servers in the lab: 1. A recursive server that's also authoritative for example.com and 2. an non recursive server that's authoritative for foo.example.com. The zone foo.example.com was delegated to server number 2. I added  multiple sample records to both zones including a.foo.EXAMPLE.COM. I first started BIND on server number 2. Upon starting BIND on server number 1, the first query I looked up was a.foo.EXAMPLE.COM. All other responses served from cache had EXAMPLE and COM in upper case. 

I will use your responses along with my lab findings to convince the application owners that they must update their code in order to prevent issues in the future.