Enhancing Subdomain Enumeration - ENTs and NOERROR
Overview
One of the most relevant techniques during the reconnaissance phase of an engagement is subdomain enumeration. Various tools, blogposts and talks covered different ways of this technique in the past. This post won't get much into detail what types exist and how subdomain enumeration could be done in general. Instead, this post aims to enhance subdomain enumeration by including a special DNS node that is often ignored.
Generally two types of subdomain enumeration exist:
- Passive subdomain enumeration
- Active subdomain enumeration
Passive subdomain enumeration could be performed by querying public information that is available in databases like censys.io, crt.sh etc. It is completely silent towards the target, since no DNS requests are sent at all.
In active subdomain enumeration, DNS queries are sent towards the nameserver of the target, in order to construct a list of valid subdomains. Simple queries like AXFR for DNS Zone-Transfer target misconfigured DNS servers. Another active enumeration technique is called subdomain bruteforce, where large lists of subdomains are prepended to the target domain and sent to the resolver in order to retrieve DNS Resource Records (RR) like 'A' for IPv4 addresses, 'CNAME' for aliases or 'AAAA' for IPv6 addresses. Further resource record types exist which however are not covered in this post.
While most tools that perform subdomain bruteforce focus on retrieving DNS Resource Records, a special type of node called "Empty Non-Terminal" (ENT)[1][2][3] exists within DNS that is rarely recognized by common enumeration tools. ENTs could be very helpful to find subdomains that might have remained hidden, when focusing on RRs only.
Relevant status codes
DNS defines a few status codes that indicate whether a DNS query was answered successfully. For everything related to this article, only the following two status codes are relevant:
- NXDOMAIN
- NOERROR
NXDOMAIN indicates that the requested domain name either does not exist at all, or the name server is not aware of its existence.
NOERROR is returned if the requested domain name is present within the DNS tree, no matter if the requested DNS Record Type exists. Alternatively the status might appear as NODATA, when the response code is NOERROR and no RR are set.
Empty Non-Terminals - ENT
Since DNS is strictly hierarchical, it is not allowed to have gaps between root and leaf node.
In the following examples, we control the domain "exampledomain.test". If we wanted to add subdomains, there are two options:
- Add the subdomain to a separate zone, handled by a different nameserver
- Add the subdomain to the zone of exampledomain.test
In this example, we go for the second option, which means that we simply modify the zonefile of exampledomain.test. This change consists of a new A-Record, referring to "blog.dev.exampledomain.test" and resolves to the IPv4 address 192.168.10.5. The resulting zonefile is shown below:
;## internal authoritative zone
$ORIGIN internal.
$TTL 86400
@ IN SOA ns.test. webmaster.test. (
2011100501 ; serial
28800 ; refresh
7200 ; retry
86400 ; expire
86400 ; min TTL
)
NS ns.test.
exampledomain IN A 192.168.10.1
www.exampledomain IN A 192.168.10.1
blog.dev.exampledomain IN A 192.168.10.5
Now that the zonefile is in place, we execute a few queries against the DNS server, starting with exampledomain.test:
dig @172.17.0.2 exampledomain.test
; <<>> DiG 9.16.1-Ubuntu <<>> @172.17.0.2 exampledomain.test
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 63605
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 1, ADDITIONAL: 1
;; WARNING: recursion requested but not available
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;exampledomain.test. IN A
;; ANSWER SECTION:
exampledomain.test. 86400 IN A 192.168.10.1
;; AUTHORITY SECTION:
internal. 86400 IN NS ns.test.
;; Query time: 0 msec
;; SERVER: 172.17.0.2#53(172.17.0.2)
;; WHEN: Mi Mai 18 10:08:38 CEST 2022
;; MSG SIZE rcvd: 84
The query above returns NOERROR, which is the status code that indicates that the request was successful. Furthermore, the A record for exampledomain.test is sent back as a response. So far, this result is as expected.
Next, the subdomain blog.dev.exampledomain.test is queried:
dig @172.17.0.2 blog.dev.exampledomain.test
; <<>> DiG 9.16.1-Ubuntu <<>> @172.17.0.2 blog.dev.exampledomain.test
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 62456
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 1, ADDITIONAL: 1
;; WARNING: recursion requested but not available
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;blog.dev.exampledomain.test. IN A
;; ANSWER SECTION:
blog.dev.exampledomain.test. 86400 IN A 192.168.10.5
;; AUTHORITY SECTION:
internal. 86400 IN NS ns.test.
;; Query time: 0 msec
;; SERVER: 172.17.0.2#53(172.17.0.2)
;; WHEN: Mi Mai 18 10:21:48 CEST 2022
;; MSG SIZE rcvd: 93
Like in the previous example, the status of the query is NOERROR, which indicates that the node exists. Furthermore the A record for the new subdomain is returned as well.
Now a third query is executed - this time against dev.exampledomain.test:
dig @172.17.0.2 dev.exampledomain.test
; <<>> DiG 9.16.1-Ubuntu <<>> @172.17.0.2 dev.exampledomain.test
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 21838
;; flags: qr aa rd; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1
;; WARNING: recursion requested but not available
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;dev.exampledomain.test. IN A
;; AUTHORITY SECTION:
internal. 86400 IN SOA ns.test. webmaster.test. 2011100501 28800 7200 86400 86400
;; Query time: 0 msec
;; SERVER: 172.17.0.2#53(172.17.0.2)
;; WHEN: Mi Mai 18 10:22:33 CEST 2022
;; MSG SIZE rcvd: 104
The status of this query is NOERROR, like in the previous examples, although this subdomain was not explicitely added to the DNS zonefile. Normally, for non-existing nodes the DNS server would return NXDOMAIN instead of NOERROR, however since DNS is hierarchical, all nodes between a leaf and the root node have to exist - no gaps are allowed.
In this case the leaf node is blog.dev.exampledomain.test - also known as a terminal, because no further nodes exist below this node.
The image below illustrates the DNS hierarchy:
The node dev.exampledomain.test is located on the path between exampledomain.test and blog.dev.exampledomain.test. The node dev was not explicitely defined within the zone file and added implicitely by the DNS server. Since no record types were defined for this node, it's called an ENT / Empty Non-Terminal.
The overall consensus that ENTs "exist" wasn't given for a long time: Although RFC1034 section 4.3.3 [1] (published in 1987) states that wildcards do not apply "When the query name or a name between the wildcard domain and the query name is know to exist. [...]" it was never defined what "exist" really means.
This issue was resolved with further clarification in RFC8020 section 3.1 [2], stating: "ENTs are nodes in the DNS that do not have resource record sets associated with them but have descendant nodes that do. The correct response to ENTs is NODATA (i.e., a response code of NOERROR and an empty answer section)"
Another clarification was provided in RFC4592 section 2.2.2 [3]: "The parenthesized 'which may be empty' specifies that empty non-terminals are explicitly recognized and that empty non-terminals 'exist'."
If all descendants of an ENT are deleted, the ENT node itself is deleted as well, as per RFC2136 section 7.16 [4]: "There is no provision for empty terminal nodes -- so if all RRs of a terminal node are deleted, the name is no longer in use, and queries of any type for that name will result in an NXDOMAIN response."
Why ENTs matter
Even if an ENT does not contain record types like A, AAAA, CNAME etc., it still provides valuable information about the hierarchy of a DNS zone.
Assuming we discover a domain like dev.exampledomain.test, and the DNS server returns NOERROR for this domain, what implications does that have?
- exampledomain.test exists
- RRs might exist for dev.exampledomain.test
- Further subdomains might exist below dev.exampledomain.test
While the first implication is trivial, the second implication indicates that other record types might exist. That means if a query for A records in dev.exampledomain.test is sent to the server and no records are present, but NOERROR is returned, it could be worth checking other common record types for this domain as well (A, AAAA, CNAME, SRV, NS, MX, TXT, DS, etc.).
The third implication is a very interesting one: If no RR are defined for this node, the node definitely contains further nodes within the DNS hierarchy if the DNS server hosting the requested zone is compliant to the RFCs. In our example the subdomain blog exists below dev.exampledomain.test. We know that because we control the zone file.
How could this be leveraged when performing reconnaissance and active subdomain enumeration?
All non-terminal nodes are by design parent to other nodes and could lead to the discovery of new subdomains. While this can be easily done with most tools for Non-Empty Non-Terminals that resolve to an IP address(carrying an A, AAAA or CNAME record), ENTs are ignored by most tools, since they don't contain common RR like A, AAAA or CNAME. Nevertheless ENTs are parent to further nodes within the DNS tree and are too valuable to be ignored.
The following image illustrates the same zone, but without knowledge about the zone layout:
Here we assume that the nodes "www", "dev" and "blog.dev" are already known. If the DNS zone is not known, each node could theoretically contain other nodes and should be inspected further.
A leaf-node (or terminal) does not contain nodes by itself and it is not allowed to be empty[5].
In other words: If active discovery techniques like subdomain bruteforce reveal a node without any RR, by definition the node can't be a leaf node, since leaf nodes are not allowed to be empty.
However if a node is discovered that contains RR, it is not possible to determine, whether the node is a non-terminal or a leaf node.
Scanning technique and support by tools
In order to enhance the process of subdomain enumeration and achieve a good coverage, the focus should be on the DNS response code rather than on resolving A, AAAA or CNAME records only.
The following steps describe a process to recursively enumerate subdomains, starting with domain.tld:
- Add domain.tld to the queue
- Perform subdomain bruteforce for next item in queue and add subdomains with DNS response code NOERROR (or NODATA) to the queue
- Remove scanned domain from queue
- If queue not empty, goto step 2
There are some caveats that should be taken into account when looking for NOERROR or NODATA responses:
Technically NODATA is not a dedicated response code, but applies when the response code is NOERROR and no RRs are defined for the node. In environments in which DNSSEC is used, NODATA responses contain NSEC or NSEC3 records, instead of being empty. NSEC and NSEC3 records disclose information about the zone and could be used for reconnaissance as well (NSEC-Walking), however this is beyond the scope of this post.
Another caveat is the concept of "Black Lies", introduced by Cloudflare in DNSSEC environments, that turn NXDOMAIN responses into NOERROR or NODATA responses[6]. Consequently it is not possible to distinguish between ENTs and leaf nodes within DNSSEC enabled zones, hosted by Cloudflare. Furthermore, DNS server implementations that deviate from the standards might also return NOERROR instead of NXDOMAIN.
Unfortunately most tools focus on resolving A, AAAA or CNAME records and would discard valid subdomains that return NOERROR and don't contain any RRs that resolve to an IP address or hostname.
The following list shows common tools for subdomain bruteforce and whether the tools are capable of filtering for DNS response codes:
- amass: no
- dnsrecon: no
- dnsx: yes
- findomain: no
- gobuster: no
- Lepus: no
- Massdns: yes
- Puredns: no
- Sublist3r: no
Massdns
Massdns[7] supports different output formats and options, as shown in the following excerpt:
[...]
Output flags:
L - domain list output
S - simple text output
F - full text output
B - binary output
J - ndjson output
Advanced flags for the domain list output mode:
0 - Include NOERROR replies without answers.
[...]
In order to detect ENTs and Non-Empty Non-Terminals without A, AAAA or CNAME records, the flag "0" should be used, since it includes responses with the NOERROR code. Unfortunately it can only be combined with the output format "L - domain list output".
In order to run massdns with the option above, the following command could be used:
massdns -r resolvers.txt urls.txt -o L0 -w urls_resolved.txt
dnsx
dnsx[8] is very efficient when it's chained with other reconnaissance tools like subfinder. Besides its ability to detect wildcards, it supports the flag -rcode that includes responses with a certain response code. Since we are interested in all NOERROR responses, the following command could be used:
./dnsx -v -w subdomains.txt -d <domain> -rcode noerror
Conclusion
Most posts and tools that deal with active subdomain enumeration, or subdomain bruteforce in particular focus on resolving A, AAAA and CNAME records and tend to discard nodes without RRs, or with RRs like SRV, TXT, etc. Consequently some subdomains and therefore potentially interesting targets might be overlooked during the reconnaissance phase.
To enhance the process of subdomain bruteforce, it should be checked whether the DNS resolver responds with NOERROR. If that is the case the according subdomain should be added to the list of targets. For these targets, all RRs should be queried to determine whether the node is empty or not.
Empty nodes are Non-Terminals by design and a separate bruteforcing process should be started for these nodes, in order to find further nodes below it. If the node is not empty, it still makes sense to query all RRs to get a good overview.
If time allows, separate bruteforcing processes could be launched against non-empty nodes, however since it's not possible to distinguish between leaf nodes and non-empty non-terminals, most of the effort might be a waste of time without knowing it.
References
[1]: https://www.rfc-editor.org/rfc/rfc1034#section-4.3.3
[2]: https://www.rfc-editor.org/rfc/rfc8020#section-3.1
[3]: https://www.rfc-editor.org/rfc/rfc4592#section-2.2.2
[4]: https://www.rfc-editor.org/rfc/rfc2136#section-7.16
[5]: https://www.rfc-editor.org/rfc/rfc4592#section-2.2.3
[6]: https://blog.cloudflare.com/black-lies/
