異常SMTP訊務(wù)與Email

1、1 1異常異常SMTPSMTP訊務(wù)與訊務(wù)與Email SpamEmail Spam的自動通告的自動通告 中央大學(xué)中央大學(xué) 電算中心電算中心 楊素秋楊素秋 Email: Email: .tw 2 2大大 綱綱 1.1.研究動機研究動機 2.2.異常異常SMTPSMTP訊務(wù)的監(jiān)測訊務(wù)的監(jiān)測 3.3.SpamSpam與異常與異常SMTPSMTP訊務(wù)的相關(guān)訊務(wù)的相關(guān) 4.4.Spam Spam 事件的自動通告事件的自動通告 5.5.結(jié)論結(jié)論3 31.1.研究動機研究動機加速加速 Email Spam Email Spam 通告通

2、告 IP IP 管理資訊查詢管理資訊查詢區(qū)網(wǎng)區(qū)網(wǎng) Routing Table Routing Table RWhoisRWhois查詢服務(wù)查詢服務(wù)Spam event Spam event 的自動通告的自動通告異常異常SMTPSMTP訊務(wù)的監(jiān)測訊務(wù)的監(jiān)測Flow count Flow count 超量超量Packet DensityPacket Density分析超量分析超量SMTPSMTP傳訊主機與通告?zhèn)饔嵵鳈C與通告spam relay/sender spam relay/sender 的相關(guān)的相關(guān)4 42.2.SMTPSMTP與與 SpamSpam傳訊傳訊SMTP SMTP 傳輸傳輸Cli

3、entClient詢問詢問DNS MX list,DNS MX list,建立信件建立信件delivery routedelivery route紀錄紀錄sendersender與與receiverreceiver間的多個間的多個mail relay/servermail relay/server將將 reverse-pathreverse-path加入加入mail headermail header與與SMTP relaySMTP relay建立雙向連接建立雙向連接, ,沿沿SMTP routeSMTP route傳送信件傳送信件relayrelay收進信件後收進信件後與下一與下一relay

4、 relay 建立連接建立連接/ /轉(zhuǎn)送信件轉(zhuǎn)送信件. .最後的最後的deliver relaydeliver relay將信件分送到用戶將信件分送到用戶mailbox. mailbox. 5 5SpamSpamUCE (Unsolicited Commercial Mail) UCE (Unsolicited Commercial Mail) spammerspammer利用自動搜尋程式利用自動搜尋程式持續(xù)尋找持續(xù)尋找 newsgroup (BBS boards)newsgroup (BBS boards)Join mailing listJoin mailing list網(wǎng)頁的網(wǎng)頁的mai

5、l addressesmail addresses所侵入系統(tǒng)的所侵入系統(tǒng)的mail accountmail accountRegular sequence mail accountRegular sequence mail account重複重複/ /密集寄送廣告信件密集寄送廣告信件 6 6SpammerSpammer以最低的成本以最低的成本, ,透過全球網(wǎng)路傳送超大量廣告信透過全球網(wǎng)路傳送超大量廣告信InternetInternet用戶用戶花費可觀的連線費用花費可觀的連線費用, ,時間與精力下載時間與精力下載/ /收取收取/ /刪除刪除大量大量spam.spam.ISPISP耗費更龐大的網(wǎng)路

6、與系統(tǒng)資源重複傳送耗費更龐大的網(wǎng)路與系統(tǒng)資源重複傳送junk mailsjunk mails影響影響mailmail的正常收送的正常收送 7 7為避免回覆大量的為避免回覆大量的spam complainspam complainSpammerSpammer藉由自動搜尋程式藉由自動搜尋程式尋找未設(shè)防的尋找未設(shè)防的SMTP server SMTP server 作為作為spam spam relay/senderrelay/sender傳送廣告信件往蒐集的傳送廣告信件往蒐集的newsgroup/mailing listnewsgroup/mailing list及及mail accountsmai

7、l accounts甚至透過甚至透過mailmail夾檔散播病蟲或攻擊程式夾檔散播病蟲或攻擊程式侵入網(wǎng)路主機侵入網(wǎng)路主機. .集結(jié)更大量的感染主機集結(jié)更大量的感染主機寄發(fā)寄發(fā)/ /轉(zhuǎn)送更大量的轉(zhuǎn)送更大量的spam. spam. 8 8減緩減緩SpamSpam倍數(shù)成長的主要途徑倍數(shù)成長的主要途徑(1)(1)回報回報/ /檢舉檢舉Spam eventSpam event減少一個減少一個 spam relay/senderspam relay/sender減少減少millions of spamsmillions of spams(2)(2)監(jiān)測可能的監(jiān)測可能的spammerspammer主機及訊務(wù)

8、主機及訊務(wù)SMTPSMTP訊務(wù)量測訊務(wù)量測篩選異常訊務(wù)量篩選異常訊務(wù)量9 9回報回報/ /檢舉檢舉Spam eventSpam event連網(wǎng)中心建立連網(wǎng)中心建立abuse Emailabuse Email帳號帳號abusedomain, spamdomain, securitydomainabusedomain, spamdomain, securitydomain接受所轄接受所轄IPIP主機的主機的Spam/ JunkSpam/ Junk通告信通告信. . 網(wǎng)路用戶網(wǎng)路用戶依據(jù)依據(jù)spam route,spam route,萃取發(fā)送主機與萃取發(fā)送主機與relay serversrelay

9、servers “Received:”, “From:” “Received:”, “From:” 紀錄項紀錄項回應(yīng)給發(fā)信主機與回應(yīng)給發(fā)信主機與relay serverrelay server擁有者擁有者ReportReport給給spam report sitespam report siteEX: EX: 1010偵測可能的偵測可能的spammerspammer主機及訊務(wù)主機及訊務(wù)依據(jù)依據(jù)Spam Spam 傳訊特徵傳訊特徵, ,實作異常實作異常SMTPSMTP訊務(wù)的統(tǒng)計訊務(wù)的統(tǒng)計IntensiveIntensive Obviously high SMTP connection count

10、Obviously high SMTP connection countIteration Iteration last for several hours last for several hours協(xié)助管理者監(jiān)測異常的協(xié)助管理者監(jiān)測異常的mailmail訊務(wù)訊務(wù)據(jù)以據(jù)以Check /var/log/maillogCheck /var/log/maillog據(jù)以據(jù)以Check user mailbox Check user mailbox 預(yù)先發(fā)現(xiàn)感染主機預(yù)先發(fā)現(xiàn)感染主機, , 通告用戶修補漏洞通告用戶修補漏洞1111通告的通告的Email Spam (2003Email Spam (200

11、3年年 7 7月至月至 1111月月) )桃園區(qū)網(wǎng)每月處理的桃園區(qū)網(wǎng)每月處理的Spam mailSpam mail通告主機總數(shù)通告主機總數(shù). .主要的主要的abuseabuse通告信件通告信件SS 通報通報廣告郵件的廣告郵件的 relay server/senderrelay server/sendermyNetWatchmyNetWatch 通報通報CodeRed/NimdaCodeRed/Nimda感染主機感染主機(80/TCP)(80/TCP)SYN Flooding (445/TCP, 17300/TCP, SYN Flooding (445/TCP, 17300/TCP, ) ) 環(huán)

12、球或派拉蒙製片環(huán)球或派拉蒙製片通告侵犯智財權(quán)的通告侵犯智財權(quán)的eDonkeyeDonkey主機及其影片檔存放主機及其影片檔存放 Others Others 1212Table 1 通告的區(qū)網(wǎng)Abuse主機數(shù)分布 Spam Spam HostsHosts SYN SYN FloodingFloodingInfringerInfringerHostsHostsJul Jul 5 518186 6Aug Aug 151522225 5Sep Sep 20200 09 9OctOct11113 36 6NovNov7 71 1121213133 3異常異常SMTPSMTP訊務(wù)的監(jiān)測訊務(wù)的監(jiān)測異常異常S

13、MTPSMTP訊務(wù)的監(jiān)測訊務(wù)的監(jiān)測SpamSpam傳訊特徵傳訊特徵IntensiveIntensiveObviously high frequency of SMTP connectionsObviously high frequency of SMTP connectionsIteration Iteration Last for Many hours Last for Many hours Mean Packet sizeMean Packet sizeLittle than 100 Bytes per PacktLittle than 100 Bytes per PacktMore th

14、an 100 Bytes per packetMore than 100 Bytes per packet1414Transportation Traffic Logs Transportation Traffic Logs all network operators depend on the all network operators depend on the quantifiable traffic log data to quantifiable traffic log data to evaluate the network performanceevaluate the netw

15、ork performanceTCPDUMPTCPDUMPNetFlow, sFlowNetFlow, sFlowOthers Others 1515TcpdumpTcpdumpa raw packet capture program.a raw packet capture program.Gather the layer 4 transportation traffic logs Gather the layer 4 transportation traffic logs throughthroughThe dump transport traffic logs involved The

16、dump transport traffic logs involved the the detail fields of each IP packet headerdetail fields of each IP packet headersource/destination IP addresses, source/destination IP addresses, source/destination application ports,source/destination application ports,protocol identity,protocol identity,num

17、ber of packets, number of packets, number of bytes, number of bytes, TCP operatorsTCP operators1616NetflowNetflow router router 轉(zhuǎn)送訊務(wù)紀錄轉(zhuǎn)送訊務(wù)紀錄Flow-based layer 4 transport traffic logFlow-based layer 4 transport traffic logSource & destination IP addressSource & destination IP addressSource &am

18、p; destination application port Source & destination application port Source & destination interface#Source & destination interface#protocol identifierprotocol identifierpacket countpacket countbyte countbyte count1717利用利用NetflowNetflow log log統(tǒng)計區(qū)網(wǎng)的異常統(tǒng)計區(qū)網(wǎng)的異常SMTPSMTP訊務(wù)訊務(wù)Accumulate SMTP se

19、rv_flowAccumulate SMTP serv_flow connection counts connection counts statisticsstatisticsNetflowlogNetflowlog gathered from router of aggregate gathered from router of aggregate networknetworkThreshold_100_flowThreshold_100_flow Less than 100 connections: 99.72 %Less than 100 connections: 99.72 % Mo

20、re than 100 connections: 0.28 %More than 100 connections: 0.28 %Threshold_30_flowThreshold_30_flow Less than 30 connections: 98.61 %Less than 30 connections: 98.61 %1818Table 2. 區(qū)網(wǎng)的SMTP Flows 特徵項分布Smtp_flowSmtp_flowcountcountFlow #/RatioFlow #/RatioByte Byte RatioRatio1 101 10136003 (94.78 %)136003

21、(94.78 %)73.1 %73.1 %11 30 11 30 5502 (3.83 %)5502 (3.83 %)12.5 %12.5 %31 70 31 70 1370 (0.95 %)1370 (0.95 %)8.1 %8.1 %71 100 71 100 231 (0.16 %)231 (0.16 %)1.1 %1.1 %101 200 101 200 226 (0.16 %)226 (0.16 %)1.2 %1.2 %201 1000201 1000145 (0.10 %)145 (0.10 %)1.8 %1.8 % 1000 100015 (0.01 %)15 (0.01 %)2

22、.2 %2.2 %1919SMTPSMTP訊務(wù)的統(tǒng)計訊務(wù)的統(tǒng)計/ /監(jiān)測監(jiān)測Monitor Abnormal SMTP Traffic of Monitor Abnormal SMTP Traffic of smtp_flowsmtp_flowi iCombine Several NetFlowCombine Several NetFlow features features SMTP service port & Src_IP & Dst_IPSMTP service port & Src_IP & Dst_IPsrc_IPsrc_IPdst_IP.(25)d

23、st_IP.(25)src_IP.(25)dst_IPsrc_IP.(25)dst_IP2020統(tǒng)計統(tǒng)計/ / 監(jiān)測異常的監(jiān)測異常的 SMTP SMTP 訊務(wù)訊務(wù)累計累計SMTP SMTP 訊務(wù)變量訊務(wù)變量透過透過 IP protocol_idIP protocol_id & application port & application port的比對的比對, ,累計累計 flowflowsmtp_flowsmtp_flowi i pktpktsmtp_flowsmtp_flowi i bytebytesmtp_flowsmtp_flowi i 排序排序/ /篩選超量的篩選超量

24、的syn_flowssyn_flows訊務(wù)訊務(wù)Monitoring SMTP TrafficMonitoring SMTP Traffic PHP + ApachePHP + Apache2121222223232424Nov 3 20:25:58 smtp3 sendmail7645: ID 801593 hA3CPot1007645:from=, size=64607, class=0, nrcpts=1,msgid=, proto=SMTP,daemon=MTA, relay=53Nov 3 20:25:58 smtp3 sendmail764

25、5: ID 801593 hA3CPot1007645:to=, delay=00:00:06, mailer=relay, pri=30258,stat=queuedNov 3 20:26:45 smtp3 mailscanner3948: Virus W32/Yaha-P found infile ./hA3CPot1007645/disney.zip/DOCUME1DennisLOCALS1Tempsetup.exeNov 3 20:26:51 smtp3 sendmail7958: ID 801593 hA3CPot1007645:to=, de

26、lay=00:00:59, xdelay=00:00:00, mailer=relay,pri=120258, relay=9 9, dsn=2.0.0, stat=Sent(hA3CP8k1016181 Message accepted for delivery)Nov 3 20:27:00 smtp3 mailscanner3948: Virus W32/Yaha-P found infile ./hA3CPot1007645/disney.zip/DOCUME1DennisLOCALS1Tempsetup.exe252526262727sy

27、slog:Oct 26 08:24:25 smtp3 sendmail13433: ID 801593 h9Q0ON2a013433: from=, size=6998, class=0, nrcpts=1, sgid=, proto=SMTP, daemon=MTA, relay= 1 (may be forged)syslog:Oct 26 08:24:25 smtp3 sendmail13425: ID 801593 h9Q0ON2a013425: from=, size=6994, class=0, nrcpts=1, sg

28、id=, proto=SMTP, daemon=MTA, relay= 5 (may be forged)syslog:Oct 26 08:24:25 smtp3 sendmail13435: ID 801593 h9Q0ON2a013435: from=, size=6971, class=0, nrcpts=1, sgid=, proto=SMTP, daemon=MTA, relay= 1 (may be forged)syslog:Oct 26 08:24:25 smtp3 sendmail13432: ID 801593

29、 h9Q0ON2a013432: from=, size=6995, class=0, nrcpts=1, sgid=, proto=SMTP, daemon=MTA, relay= 4 (may be forged)syslog:Oct 26 08:24:25 smtp3 sendmail13434: ID 801593 h9Q0ON2a013434: from=, size=6965, class=0, nrcpts=1, 2828Mail Relay TestingMail Relay Testingmrtmrtftp:/ v

30、 test.patterns test.message host_ip_add./mrt v test.patterns test.message host_ip_add2929 ann# ./mrt -v ./test.patterns ./test.message 45mrt: 45: Error connecting: Connection refusedmrt: 45: Error connecting: Connection refusedmrt: 45: Error connecting

31、: Connection refusedmrt: 45: Error connecting: Connection refusedmrt: 45: Error connecting: Connection refusedmrt: 45: Error connecting: Connection refusedmrt: 45: Error connecting: Connection refusedmrt: 45: Error connecting: Connection re

32、fusedmrt: 45: Error connecting: Connection refusedmrt: 45: Error connecting: Connection refusedmrt: 45: Error connecting: Connection refusedmrt: 45: Error connecting: Connection refusedmrt: 45: Error connecting: Connection refusedmrt: 163.2

33、5.121.245: Error connecting: Connection refusedmrt: 45: Error connecting: Connection refusedmrt: 45: Error connecting: Connection refusedmrt: 45: Error connecting: Connection refused3030ann# ./mrt -v ./test.patterns ./test.message mrt: : Mess

34、age acceptedmrt: : Message acceptedmrt: : Message acceptedmrt: : SMTP error (553) reading MAIL responsemrt: : Message acceptedmrt: : Message acceptedmrt: : Message acceptedmrt: : Message acceptedmrt: : Message ac

35、ceptedmrt: : Message acceptedmrt: : Message acceptedmrt: : Message acceptedmrt: : Message acceptedmrt: : Message acceptedmrt: : Message acceptedmrt: : Message acceptedmrt: : Message acceptedmrt: : SMTP

36、 error (553) reading MAIL response3131ann# ./mrt -v ./test.patterns ./test.message 28mrt: 28: SMTP error (550) reading RCPT responsemrt: 28: SMTP error (550) reading RCPT responsemrt: 28: SMTP error (550) reading RCPT responsemrt: 28: SMTP

37、error (550) reading RCPT responsemrt: 28: SMTP error (550) reading RCPT responsemrt: 28: SMTP error (550) reading RCPT responsemrt: 28: SMTP error (550) reading RCPT responsemrt: 28: SMTP error (553) reading RCPT responsemrt: 28: SMTP error

38、 (553) reading RCPT responsemrt: 28: SMTP error (553) reading RCPT responsemrt: 28: SMTP error (550) reading RCPT responsemrt: 28: SMTP error (550) reading RCPT responsemrt: 28: SMTP error (550) reading RCPT responsemrt: 28: SMTP error (550

39、) reading RCPT responsemrt: 28: SMTP error (550) reading RCPT responsemrt: 28: SMTP error (550) reading RCPT responsemrt: 28: SMTP error (550) reading RCPT response3232數(shù)據(jù)分析數(shù)據(jù)分析60 % 60 % 通告通告spam relay/senderspam relay/sender可由統(tǒng)計的異?？捎山y(tǒng)計的異常SMTPSMTP主機佇列中篩選得主機佇列中篩選得7

40、7月份的月份的60%60%8 8月份的月份的60% 60% 9 9月份的月份的60%60%1010月份的月份的100%100%1111月份的月份的100%100%異常異常SMTP/SYN FloodingSMTP/SYN Flooding訊務(wù)監(jiān)測訊務(wù)監(jiān)測發(fā)現(xiàn)發(fā)現(xiàn)Spam & Spam & 網(wǎng)路侵擾訊務(wù)網(wǎng)路侵擾訊務(wù)3333Table 2 區(qū)網(wǎng)Abuse host分布(2003年)Abnormal SMTP TrafficAbnormal www /SYN FloodingJul 60 %60 %43 %43 %Aug 60 % 60 % 48 %48 %Sep 60 % 60 %

41、 - -Oct 55 % 55 %100 %100 %Nov 100 % 100 %100 %100 %34344 Spam 事件的自動通告 Spam/Spam/攻擊訊務(wù)通告事件攻擊訊務(wù)通告事件倍數(shù)成長的倍數(shù)成長的spam spam 通告通告超量的異常超量的異常 SMTP Traffic SMTP Traffic 網(wǎng)路管理者網(wǎng)路管理者非常依賴非常依賴IPIP管理資訊查詢系統(tǒng)管理資訊查詢系統(tǒng)通告感染主機用戶與管理者通告感染主機用戶與管理者, ,修補系統(tǒng)修補系統(tǒng)自動阻斷攻擊訊務(wù)自動阻斷攻擊訊務(wù), ,防堵攻擊訊務(wù)的持續(xù)擴散防堵攻擊訊務(wù)的持續(xù)擴散3535spam mailspam mail的自動通告系

42、統(tǒng)的自動通告系統(tǒng)自動自動Query Query IPIP管理資訊管理資訊,Email,Email通告通告藉由藉由SNMP pulling router ipRouteSNMP pulling router ipRoute MIB, MIB, 快速萃取連網(wǎng)的龐大快速萃取連網(wǎng)的龐大 routingrouting資訊資訊建立建立IPIP管理資訊查詢服務(wù)管理資訊查詢服務(wù)依據(jù)依據(jù) NextHopNextHop integrate integrate The extracted Routing TableThe extracted Routing Table 連線單位通訊資訊檔連線單位通訊資訊檔 RWho

43、isRWhois IP IP管理資料庫管理資料庫 3636ipRouteipRoute SNMP MIB SNMP MIB儲存連網(wǎng)單位的儲存連網(wǎng)單位的routing routing 資訊資訊Network address Network address NetMaskNetMask辨識號辨識號 .....1.11NextHopNextHop 辨識號辨識號 .....1.7Mansfield G. Mansfield G. 曾藉由曾藉由ipRouteipRoute MIB M

44、IB重複搜尋各層重複搜尋各層routers ipRouterouters ipRoute MIB MIB 自動構(gòu)建區(qū)域網(wǎng)路拓樸自動構(gòu)建區(qū)域網(wǎng)路拓樸 3737重複萃取網(wǎng)段重複萃取網(wǎng)段IPIP位址與對應(yīng)的位址與對應(yīng)的NetMask/ NextHopNetMask/ NextHop位址位址分別以分別以IPIP網(wǎng)段位址網(wǎng)段位址index,index,儲存儲存NetMaskNetMask List ListNextHopNextHop List. List. 結(jié)合結(jié)合NetMask ,NextHopNetMask ,NextHop 與與SegmentSegment佇列佇列快速重建龐大的區(qū)網(wǎng)快速重建龐大的

45、區(qū)網(wǎng)ip_routingip_routing 紀錄存檔紀錄存檔3838ipRouteMask OIDip.ipRouteTable.ipRouteEntry.ipRouteMask. = IpAddress: ip.ipRouteTable.ipRouteEntry.ipRouteMask. = IpAddress: ip.ipRouteTable.ipRouteEntry.ipRouteMask. = IpAddress: ip.ipRoute

46、Table.ipRouteEntry.ipRouteMask. = IpAddress: ipRouteNextHop OID ip.ipRouteTable.ipRouteEntry.ipRouteNextHop. = IpAddress: 2ip.ipRouteTable.ipRouteEntry.ipRouteNextHop. = IpAddress: 11ip.ipRouteTable.ipRouteEntry.ipRouteNextHop.19

47、 = IpAddress: 16ip.ipRouteTable.ipRouteEntry.ipRouteNextHop. = IpAddress: 113939NextHop Dest. Netmask Seg =26, ,, 256 , , , 256 1, , , 256 37, 140

48、.138.0.0, , 25609, , , 409, , , 44040IPIP邏輯位址不包含任何管理資訊邏輯位址不包含任何管理資訊RouterRouter藉由藉由routing tablerouting table的查詢的查詢依據(jù)依據(jù) NextHopNextHop 紀錄紀錄 switch packetswitch packetSwitch Switch 往正確的往正確的 routing interfacerouting interf

49、ace4141RWhoisRWhois分享軟體分享軟體利用利用Mark KostersDataBaseMark KostersDataBase (MKDB) (MKDB) 支援資支援資料的管理與查詢料的管理與查詢. .資料庫查詢伺服程式資料庫查詢伺服程式rwhoisdrwhoisd資料庫建置程式資料庫建置程式rwhoisd_indexerrwhoisd_indexer4242RWhoisRWhois Server Server藉由藉由IPIP管理資料庫伺服系統(tǒng)的建置管理資料庫伺服系統(tǒng)的建置, ,作為自動作為自動通告通告Spam Spam 的基礎(chǔ)的基礎(chǔ). .讀取讀取routingrouting紀錄紀錄, ,依據(jù)依據(jù)NexthopNexthop 紀錄比對紀錄比對/ /萃取對萃取對應(yīng)的管理聯(lián)絡(luò)資訊檔應(yīng)的管理聯(lián)絡(luò)資訊檔構(gòu)建構(gòu)建RWhoisRWhois network schema network schema關(guān)聯(lián)紀錄檔關(guān)聯(lián)紀錄檔建立資料庫建立資料庫indexing, index