信息收集框架——recon-ng
背景：在渗透测试前期做攻击⾯发现（信息收集）时候往往需要⽤到很多⼯具，最后再将搜集到的信息汇总到⼀块。
现在有这样⼀个现成的框架，⾥⾯集成了许多信息收集模块、信息存储数据库、以及报告⽣成模块，为⼯程化信息收集提供了可能。
它就是recon-ng。recon-ng使⽤python编写，其使⽤⽅式和metasploit⼗分相似
使⽤⽅法介绍：
1、新建⼯作区（建议⼀个渗透⽬标⼀个⼯作区，这样能确保搜集到的信息都是针对⼀个⽬标的）
命令：Recon-ng -w ⼯作区名字
例：
recon-ng -w cctv
# 通过上⾯的命令创建‘cctv’⼯作区后可以通过如下命令查看⼯作区情况
[recon-ng][cctv] > show workspaces
+------------+
| Workspaces |
+------------+
| cctv |
| default |
+------------+
2、设置搜索引擎api
Keys list ===>查看现有搜索引擎api
keys add shodan fdkasjkfljklasjkldffjalks ===>设置shodan搜索api
[recon-ng][cctv] > keys list
+--------------------------+
| Name | Value |
+--------------------------+
| bing_api | |
| builtwith_api | |
| censysio_id | |
| censysio_cret | |
| flickr_api | |
| fullcontact_api | |
| github_api | |
| google_api | |
| hashes_api | |
| ipinfodb_api | |
| ipstack_api | |
| jigsaw_api | |
| jigsaw_password | |
| jigsaw_urname | |
| pwnedlist_api | |
| pwnedlist_iv | |
| pwnedlist_cret | |
| shodan_api | |
| twitter_api | |
| twitter_cret | |
| virustotal_api | |
+--------------------------+
[recon-ng][cctv] > keys add shodan_api fdkasjkfljklasjkldffjalks
3、show options(查看全局设置)
[recon-ng][cctv] > show options
Name Current Value Required Description
---------- ------------- -------- -----------
NAMESERVER 8.8.8.8 yes namerver for DNS interrogation
PROXY no proxy rver (address:port)
THREADS 10 yes number of threads (where applicable)
TIMEOUT 10 yes socket timeout (conds)
USER-AGENT Recon-ng/v4 yes ur-agent string
VERBOSITY 1 yes verbosity level (0 = minimal, 1 = verbo, 2 = debug)
建议设置代理，让可以访问google(不得不佩服google的搜索能⼒)
t PROXY 127.0.0.1:1087
4、查询包含哪些可⽤模块
exploitation/injection/command_injector recon/domains-contacts/pen recon/domains-vulnerabilities/ghdb recon/netblocks-ports/census_2012 exploitation/injection/xpath_bruter recon/domains-contacts/pgp_arch recon/domains-vulnerabilities/punkspider recon/netblocks-ports/censysio
import/csv_file recon/domains-contacts/whois_pocs recon/domains-vulnerabilities/xsd recon/ports-hosts/migrate_ports
import/list recon/domains-credentials/pwnedlist/account_creds recon/domains-vulnerabilities/xsspod recon/profiles-contacts/dev_diver
recon/companies-contacts/bing_linkedin_cache recon/domains-credentials/pwnedlist/api_usage recon/hosts-domains/migrate_hosts recon/profiles-contacts/github_urs recon/companies-contacts/jigsaw/point_usage recon/domains-credentials/pwnedlist/domain_creds recon/hosts-hosts/bing_ip recon/profiles-profiles/namechk
recon/companies-contacts/jigsaw/purcha_contact recon/domains-credentials/pwnedlist/domain_ispwned recon/hosts-hosts/ipinfodb recon/profiles-profiles/profiler recon/companies-contacts/jigsaw/arch_contacts recon/domains-credentials/pwnedlist/leak_lookup recon/hosts-hosts/ipstack recon/profiles-profiles/twitter_mentioned recon/companies-contacts/pen recon/domains-credentials/pwnedlist/leaks_dump recon/hosts-hosts/resolve recon/profiles-profiles/twitter_mentions recon/companies-domains/pen recon/domains-domains/brute_suffix recon/hosts-hosts/rever_resolve recon/profiles-repositories/github_repos recon/companies-multi/github_miner recon/domains-hosts/bing
_domain_api recon/hosts-hosts/ssltools recon/repositories-profiles/github_commits recon/companies-multi/whois_miner recon/domains-hosts/bing_domain_web recon/hosts-hosts/virustotal recon/repositories-vulnerabilities/gists_arch recon/contacts-contacts/mailtester recon/domains-hosts/brute_hosts recon/hosts-locations/migrate_hosts recon/repositories-vulnerabilities/github_dorks recon/contacts-contacts/mangle recon/domains-hosts/builtwith recon/hosts-ports/shodan_ip reporting/csv
recon/contacts-contacts/unmangle recon/domains-hosts/certificate_transparency recon/locations-locations/geocode reporting/html
recon/contacts-credentials/hibp_breach recon/domains-hosts/findsubdomains recon/locations-locations/rever_geocode reporting/json
recon/contacts-credentials/hibp_paste recon/domains-hosts/google_site_web recon/locations-pushpins/flickr reporting/list
recon/contacts-domains/migrate_contacts recon/domains-hosts/hackertarget recon/locations-pushpins/shodan reporting/proxifier
recon/contacts-profiles/fullcontact recon/domains-hosts/mx_spf_ip recon/locations-pushpins/twitter reporting/pushpin
recon/credentials-credentials/adobe recon/domains-hosts/netcraft recon/locations-pushpins/youtube reporting/xlsx
recon/credentials-credentials/bozocrack recon/domains-hosts/shodan_hostname recon/netblocks-companies/whois_orgs reporting/xml
recon/credentials-credentials/hashes_org recon/domains-hosts/ssl_san recon/netblocks-hosts/rever_resolve
也可以通过arch命令来查找相关模块
[recon-ng][cctv] > arch google
[*] Searching for 'google'...
Recon
-----
recon/domains-hosts/google_site_web
此时⼤家可能会有疑问，这么多模块我怎么知道哪个模块是⼲什么使的呢？这个时候我们可以u相应模块后⽤show info看到关于该模块的详细解释[recon-ng][cctv] > u recon/domains-hosts/google_site_web
[recon-ng][cctv][google_site_web] > show info
Name: Google Hostname Enumerator
Path: modules/recon/domains-hosts/google_site_web.py
Author: Tim Tomes (@LaNMaSteR53)
Description:
Harvests hosts by using the 'site' arch operator. Updates the 'hosts' table with
the results.
Options:
Name Current Value Required Description
------ ------------- -------- -----------
SOURCE yes source of input (e 'show info' for details)
Source Options:
default SELECT DISTINCT domain FROM domains WHERE domain IS NOT NULL
<string> string reprenting a single input
<path> path to a file containing a list of inputs
query <sql> databa query returning one column of inputs
此外recon-ng会将收集到的信息⾃动存⼊数据库，后⾯咱们可以将这些数据掏出来进⾏⼆次查询。可以通过下⾯这个命令查看数据库有哪些表：
[recon-ng][cctv] > show schema
+---------------+
| domains |
+---------------+
| domain | TEXT |
| module | TEXT |
+---------------+
+--------------------+
| companies |
+--------------------+
| company | TEXT |
| description | TEXT |
| module | TEXT |
+--------------------+
+-----------------+
| netblocks |
+-----------------+
| netblock | TEXT |
| module | TEXT |
+-----------------+
+-----------------------+
| locations |
+-----------------------+
| latitude | TEXT |
| longitude | TEXT |
| street_address | TEXT |
| module | TEXT |
+-----------------------+
+---------------------+
| vulnerabilities |
+---------------------+
| host | TEXT |
| reference | TEXT |
| example | TEXT |
| publish_date | TEXT |
+-------------------+
| ports |
+-------------------+
| ip_address | TEXT |
| host | TEXT |
| port | TEXT |
| protocol | TEXT |
| module | TEXT |
+-------------------+
+-------------------+
| hosts |
+-------------------+
| host | TEXT |
| ip_address | TEXT |
| region | TEXT |
| country | TEXT |
| latitude | TEXT |
| longitude | TEXT |
| module | TEXT |
+-------------------+
+--------------------+
| contacts |
+--------------------+
| first_name | TEXT |
| middle_name | TEXT |
| last_name | TEXT |
| email | TEXT |
| title | TEXT |
| region | TEXT |
| country | TEXT |
| module | TEXT |
+--------------------+
+-----------------+
| credentials |
+-----------------+
| urname | TEXT |
| password | TEXT |
| hash | TEXT |
| type | TEXT |
| leak | TEXT |
| module | TEXT |
+-----------------+
+-----------------------------+
| leaks |
+-----------------------------+
| leak_id | TEXT |
| description | TEXT |
| source_refs | TEXT |
| leak_type | TEXT |
| title | TEXT |
| import_date | TEXT |
| leak_date | TEXT |
| attackers | TEXT |
| num_entries | TEXT |
| score | TEXT |
| num_domains_affected | TEXT | | attack_method | TEXT |
| target_industries | TEXT |
| password_hash | TEXT | | password_type | TEXT |
| targets | TEXT |
| media_refs | TEXT |
| module | TEXT |
+-----------------------------+
+---------------------+
| pushpins |
+---------------------+
| source | TEXT |
| screen_name | TEXT |
| profile_name | TEXT |
| profile_url | TEXT |
| media_url | TEXT |
| thumb_url | TEXT |
| message | TEXT |
| latitude | TEXT |
| longitude | TEXT |
| time | TEXT |
| module | TEXT |
+---------------------+
+-----------------+
| profiles |
+-----------------+
| urname | TEXT |
| resource | TEXT |
| url | TEXT |
| category | TEXT |
| notes | TEXT |
| module | TEXT |
+-----------------+
+--------------------+
| repositories |
+--------------------+
| name | TEXT |
| owner | TEXT |
| description | TEXT |
| resource | TEXT |
5、使⽤⽅法举例（拿搜索⼦域名与对应ip的场景来举例）
使⽤google搜索来查询⽬标有哪些⼦域名
[recon-ng][cctv] > u recon/domains-hosts/google_site_web
[recon-ng][cctv][google_site_web] > show options # 查看需要填哪些数据
Name Current Value Required Description
-
----- ------------- -------- -----------
SOURCE default yes source of input (e 'show info' for details)
[recon-ng][cctv][google_site_web] > t # 设置⽬标域名
SOURCE =>
[recon-ng][cctv][google_site_web] > run #开始运⾏
也可以使⽤暴⼒猜解的⽅式来获取⽬标⼦域名：
[recon-ng][cctv] > u recon/domains-hosts/brute_hosts
[recon-ng][cctv][brute_hosts] > show options
Name Current Value Required Description
-------- ------------- -------- -----------
SOURCE default yes source of input (e 'show info' for details)
WORDLIST /usr/local/Cellar/recon-ng/4.9.6/libexec/ yes path to hostname wordlist # 字典路径
[recon-ng][cctv][brute_hosts] > t # 设置⽬标域名
SOURCE =>
[recon-ng][cctv][brute_hosts] > run #开始运⾏
运⾏完毕后查询到的数据将⾃动存⼊数据库，我们可以通过'show hosts'或'query+sql语句'的⽅式来查询，例：
[recon-ng][cctv] > show hosts
+-----------------------------------------------------------------------------------------------------------+
| rowid | host | ip_address | region | country | latitude | longitude | module |
+-----------------------------------------------------------------------------------------------------------+
| 1 | | | | | | | google_site_web |
| 2 | | | | | | | google_site_web |
| 3 | | | | | | | google_site_web |
+-----------------------------------------------------------------------------------------------------------+
[recon-ng][cctv] >query lect * from hosts;
+-----------------------------------------------------------------------------------------------------------+
| rowid | host | ip_address | region | country | latitude | longitude | module |
+-----------------------------------------------------------------------------------------------------------+
| 1 | | | | | | | google_site_web |
| 2 | | | | | | | google_site_web |
| 3 | | | | | | | google_site_web |
+-----------------------------------------------------------------------------------------------------------+
# 为了保证隐私删掉了⼤部分数据，只给3个做为举例
数据库⾥已经有⽬标的⼦域名信息，现在想基于数据库⾥信息做进⼀步查询可以吗？当然可以，我们以查询域名对应的ip为例：
[recon-ng][cctv] > u recon/hosts-hosts/resolve
[recon-ng][cctv][resolve] > show options
Name Current Value Required Description
------ ------------- -------- -----------
SOURCE default yes source of input (e 'show info' for details) # 正常来说SOURCE后应该是跟⼀个域名信息，⽐如''
[recon-ng][cctv][resolve] > t SOURCE query lect host from hosts # 这⾥厉害了哦！我们要查的是⼀个表的内容，如果⼀个域名设置⼀次那还不累死了？ recon-ng竟然⽀持将值设为⼀个sql语句！这样就可以批量查询表内的数据了！SOURCE => query lect host from hosts
[recon-ng][cctv][resolve] > run
执⾏完成后我们可以看下现在数据库⾥的内容有什么变化：
[recon-ng][cctv][resolve] > show hosts
+----------------------------------------------------------------------------------------------------------------+
| rowid | host | ip_address | region | country | latitude | longitude | module |
+----------------------------------------------------------------------------------------------------------------+
| 1 | | 123.125.195.125 | | | | | google_site_web |
| 2 | | 114.112.172.231 | | | | | google_site_web |
| 3 | | 111.206.186.245 | | | | | google_site_web |
| 4 | | 123.125.195.125 | | | | | resolve |
| 5 | | 114.112.172.231 | | | | | resolve |
| 6 | | 111.206.186.245 | | | | | resolve |
+----------------------------------------------------------------------------------------------------------------+
# 可以看到已经把查询到的ip地址填⼊表内了
就拿我们现在查询到的数据来举例说明⼀下该怎么导出报表
[recon-ng][cctv] > arch report # 查下看有哪些报表相关模块
[*] Searching for 'report'...
Reporting
---------
reporting/csv
reporting/html
reporting/json
reporting/list
reporting/proxifier
reporting/pushpin
reporting/xlsx
reporting/xml
-------- ------------- -------- -----------
CREATOR yes creator name for the report footer
CUSTOMER yes customer name for the report header
FILENAME /Urs/liwei/.recon-ng/workspaces/cctv/results.html yes path and filename for report output # 报表导出路径 SANITIZE True yes mask nsitive data in the report
[recon-ng][cctv][html] > t CREATOR liwei # 填写报告作者
CREATOR => liwei
[recon-ng][cctv][html] > t CUSTOMER cctv # 填写⽤户单位名称
CUSTOMER => cctv
[recon-ng][cctv][html] > run
[*] Report generated at '/Urs/liwei/.recon-ng/workspaces/cctv/results.html'. # 导出成功
[recon-ng][cctv][html] >
最终报表长这样：
注：以下是引⾃⽹友对各个模块的简要说明：
cache_snoop – DNS缓存录制
interesting_files – 敏感⽂件探测
command_injector – 远程命令注⼊shell接⼝
xpath_bruter – Xpath注⼊爆破
csv_file – ⾼级csv⽂件导⼊
list – List⽂件导⼊
point_usage – Jigsaw – 统计信息提取⽤法
purcha_contact – Jigsaw – 简单的联系查询
arch_contacts – Jigsaw联系枚举
jigsaw_auth – Jigsaw认证联系枚举
linkedin_auth – LinkedIn认证联系枚举
github_miner – Github资源挖掘
whois_miner – Whois数据挖掘
bing_linkedin – Bing Linkedin信息采集
email_validator – SalesMaple邮箱验证
mailtester – MailTester邮箱验证
mangle – 联系分离
unmangle –联系反分离
hibp_breach –Breach搜索
hibp_paste – Paste搜索
pwnedlist – PwnedList验证
migrate_contacts – 域名数据迁移联系
facebook_directory – Facebook⽬录爬⾏
fullcontact – FullContact联系枚举
adobe – Adobe Hash破解
bozocrack – PyBozoCrack Hash 查询
hashes_org – Hash查询
leakdb – leakdb Hash查询
metacrawler – 元数据提取
pgp_arch – PGP Key Owner查询
salesmaple – SalesMaple联系获取
whois_pocs – Whois POC获取
account_creds – PwnedList – 账户认证信息获取
api_usage – PwnedList – API使⽤信息
domain_creds – PwnedList – Pwned域名认证获取
domain_ispwned – PwnedList – Pwned域名统计获取
leak_lookup – PwnedList – 泄露信息查询
leaks_dump – PwnedList –泄露信息获取
brute_suffix – DNS公共后缀爆破
baidu_site – Baidu主机名枚举
bing_domain_api – Bing API主机名枚举
