slow non-authoritative recursive lookups

[btb]

hello-

i have a nameserver that is performing a few different functions, and 
is very slow whenever it has to do a recursive lookup for anything.
i'm running bind 9.2.4 (debian), and have an internal and external view 
setup.  the internal view is acting as a resolver for local clients, as 
well as being authoritative for a few zones, and the external view is 
doing authoritative answers only for a few zones, and transferring 
those zones from a master.

recursive lookups are taking on the order of 5000 msec, give or take, 
until the record is in the cache.  however, if i add a forwarders 
statement to options, and tell bind to use my isp's resolvers as 
forwarders, things are nice and fast (300-400 msec consistently).  i've 
run through the logs in debug priority, and have even spent some time 
w/ tcpdump, but i am having trouble interpreting the many lines of logs 
that appear for a lookup.  the only thing that jumped out at me, was a 
lot of chatter between bind and the tld nameservers - maybe too much.  
but i'm quite green in this area, so it may be normal.

any insight on tracking down the cause is greatly appreciated!

thanks
-ben