內(nèi)容linux運(yùn)維-2014教程cache p

SquidPhpinfo()→opcodeNginxReverse

proxyLNAMP

=

Nginx

→

Apache+php+MySQL

Php:

CPU

MySQL:

CPU

Php

→

mysql.sock

MySQL

Php→

Memcached

MySQL

(query

cache)

LNAMPLNMPLAMPPHPXCacheopcodememcachelibmemcached2020/11/24Cryptographypage:2What

is

Squid?Squid的誕生Squid功能An

ever-growing

number

ofcompanies

use

Squid

tosave

on

their

internet web

traffic,

improveperformance,

deliver

faster

browsing

to

their

end-clients

and

provide

static,

dynamicand

streamingcontent

to millions

of

internet

users

worldwideSquid

的特點(diǎn)offers

a

rich

access

control,

authorizationandlogging

environment

to

develop

web

proxy

and

contentserving

applications2020/11/24Cryptographypage:3What

are

cachable

objects?An

Internet

Object

is

a

file, or

response

to

aqueryfor

an

Internet

service

such

as

FTP,

HTTP,

or

gopherNot

dnsA

client

requests

an

Internet

object

from

a

caching

proxy

if

the

object

is

not

already

cached,

the

proxy

server

fetchesthe

object

(either

from

the

host

specified

in

the

URL

or

froma

parent

or

sibling

cache)

and

delivers

it

to

the

client2020/11/24Cryptographypage:42020/11/24Cryptographypage:5ICP

protocol:

Internet

Cache

ProtocolUsed

for

communication

among

squid

cachesICP

is

primarily

used

within

a

cache

hierarchy

to

locatespecific

objects

in

sibling

cachesICP

is

currently

implemented

on

top

of

3130/UDPSquid基本配置squid包包名：squid-2.6.STABLE6服務(wù)名：squid主程序：/usr/sbin/squid配置

：/etc/squid/主配置文件：/etc/squid/squid.conf默認(rèn) 端口：TCP

3128默認(rèn) 日志文件：/var/log/squid/access.log2020/11/24Cryptographypage:6主配置文件squid.conf常用配置項(xiàng)http_port

3128cache_mem

64

MBum_object_size

4096

KBreply_body_max_size

10240000

allow

allaccess_log

/var/log/squid/access.log

squidvisible_hostname

cache_dir

ufs

/var/spool/squid

100

16

256為緩存

分配的磁盤空間（MB）緩存空間的一級(jí)子 個(gè)數(shù)緩存空間的二級(jí)子 個(gè)數(shù)緩存數(shù)據(jù)的存儲(chǔ)格式2020/11/24Cryptographypage:7exporthttp_proxy=2020/11/24Cryptographypage:82020/11/24Cryptographypage:9acl

all

/http_access

deny

allACL

控制ACL（Access

ControlList，控制列表）可以從客戶機(jī)的IP地址、請(qǐng)求 的URL/ /文件類型、時(shí)間、并發(fā)請(qǐng)求數(shù)等各方面進(jìn)行控制應(yīng)用 控制的方式定義acl列表acl

列表名稱列表類型列表內(nèi)容…針對(duì)acl列表進(jìn)行限制http_access

allow或deny

列表名……2020/11/24Cryptographypage:10ACL

控制最基本的ACL控制示例任何客戶機(jī)使用 服務(wù)acl

all

src

/http_access

deny

all2020/11/24Cryptographypage:112020/11/24Cryptographypage:12reply_body_max_size

10240000

allow

allACL

控制常用的acl列表類型srcdstportsrcdsttimemaxconnurl_regexurlpath_regex2020/11/24Cryptographypage:132020/11/24Cryptographypage:148:00-18:002020/11/24Cryptographypage:15acl

worktime

time08:00-12:00acl

worktime

time

14:00-18:00http_access

allow

worktime

mynetwork2020/11/24Cryptographypage:16arp2020/11/24Cryptographypage:172020/11/24Cryptographypage:182020/11/24Cryptographypage:192020/11/24Cryptographypage:202020/11/24Cryptographypage:212020/11/24Cryptographypage:222020/11/24Cryptographypage:232020/11/24Cryptographypage:242020/11/24Cryptographypage:252020/11/24Cryptographypage:262020/11/24Cryptographypage:272020/11/24Cryptographypage:282020/11/24Cryptographypage:292020/11/24Cryptographypage:302020/11/24Cryptographypage:312020/11/24Cryptographypage:32localnet,denytime:

18:00-23:59ACL

控制ACL列表定義示例acl

LAN1

src

/24acl

PC1

src

2/32acl

Blk_

dstacl

Work_Hours

timeMTWHF

08:30-17:30acl

Max20_Conn

maxconn

20acl

Blk_URL

url_regex

-i

^rtsp://

^mms://acl

Blk_Words

urlpath_regex

-i

sexadultacl

RealFile

urlpath_regex

-i

\.rmvb$

\.rm$2020/11/24Cryptographypage:33ACL

控制根據(jù)已經(jīng)定義的部分ACL列表進(jìn)行 控制http_access

deny

LAN1

Blk_URLhttp_access

deny

LAN1

Blk_Wordshttp_access

deny

PC1

RealFilehttp_access

deny

PC1

Max20_Connhttp_access

allow

LAN1

Work_Hours2020/11/24Cryptographypage:34ACL

控制控制規(guī)則的匹配順序沒(méi)有設(shè)置任何規(guī)則時(shí)——

將 所有客戶端的 請(qǐng)求有規(guī)則但找不到相匹配的項(xiàng)時(shí)——

將采用與最后一條規(guī)則相反的權(quán)限，即如果最后一條規(guī)則是allow，那么就

客戶端的請(qǐng)求，否則允許該請(qǐng)求2020/11/24Cryptographypage:35配置的基本條件實(shí)現(xiàn)前提：

客戶機(jī)的Web數(shù)據(jù)要能經(jīng)過(guò)服務(wù)構(gòu)建在網(wǎng)關(guān)（

）主機(jī)中配置要求：服務(wù)程序能夠支持規(guī)則，將客戶機(jī)的Web

數(shù)據(jù)自動(dòng)重定向給

服

設(shè)置務(wù)程序處理2020/11/24Cryptographypage:36配置服務(wù)的典型應(yīng)用環(huán)境Internet服務(wù)器局域網(wǎng)PC機(jī)1/24eth0:1/30eth1:/249/302020/11/24Cryptographypage:37配置基本實(shí)現(xiàn)步驟修改squid.conf配置文件，并重新加載該配置

http_port

:8080

transparent添加iptables規(guī)則

iptables

-t

nat

-I

PREROUTING

-i

eth1

-s

/24

-ptcp

--dport

80

-j

REDIRECT

--to-ports

8080客戶機(jī)瀏覽器

不需要在瀏覽器中指定 服務(wù)器的地址、端口驗(yàn)證 的實(shí)施效果2020/11/24Cryptographypage:382020/11/24Cryptographypage:39Iptables

–t

nat

–A

POSTROUTING

–s/24–j

SNAT

–to-source42020/11/24Cryptographypage:40http_port

:3128

transparent配置反向Internet反向服務(wù)器Internet中的客戶機(jī)9/30eth1:/24eth0:1/30/24/24服務(wù)器群2020/11/24Cryptographypage:41accelAccelerator

mode.

Also

needs

at

least

one

ofvhost/vport/defaultsitedefaultsite=

nameWhat

to

use

for

the

Host:

header

if

it

is

not

present

in

arequest.

Determines

what

site

(not

origin

server)

acceleratorsshould

consider

the

default.

Implies

accel.vhostAccelerator

mode

using

Host

header

for

virtualsupport.

Implies

accel.2020/11/24Cryptographypage:422020/11/24Cryptographypage:43cache_peerTo

specify

other

caches

in

a

hierarchy,

use

the

format:

cache_peer

hostname

type

http-port

icp-port

[options]For

example

cache_peer

cache_peer

cache_peer

parentsiblingsibling3128

3130

proxy-only

default3128

3130

proxy-only3128

3130

proxy-onlytype:

either

'parent',

'sibling',

or

'multicast‘proxy-port: The

port

number

where

the

cache

listens

forproxy

requestsicp-port: Used

for

queryingneighborcaches

about

objects5,2,12020/11/24Cryptographypage:44weight=nTo

affect

the

selection

of

a

peerduring

any

weighted

peer-selection

mechanismsThe

weight

must

be

an

integer;

default

is

1,

larger

weights

arefavored

moremax-conn=nTo

limit

the

amount

of

connections

Squid

may

open

to

thispeeroriginserverCauses

this

parent

peer

to

be

contacted

as

a

origin

serverMeant

to

be

used

in

accelerator

setups2020/11/24Cryptographypage:45配置反向基本實(shí)現(xiàn)步驟修改squid.conf文件，并重新加載該配置http_port

1:80

vhostcache_peer

parent

80

0

originserver

weight=5

max-conn=30cache_peer

parent

80

0

originserver

weight=5

max-conn=30cache_peer

parent

80

0

originserver

weight=5

max-conn=30cache_peer

parent

80

0

originserver

weight=1

max-conn=8cache_peer

Web服務(wù)器地址

服務(wù)器類型

http端口

icp端口

[可選項(xiàng)]memcachedtext,

binaryiptcp,

udpnet

ip

802020/11/24Cryptographypage:4710k,

34K2020/11/24Cryptographypage:484k10M已用，空閑2020/11/24Cryptographypage:4948bytes80bytes72增長(zhǎng)因子growth

factor,

1.25

48bytes:

slab

class,

slab

chunk

80bytes2020/11/24Cryptographyindex.html/42020/11/24Cryptographypage:512020/11/24Cryptographypage:52perl

modulecache::memcachedphpmemcachememcachedC/C++libmemecached

命令行工具memadmin1-2^322020/11/24Cryptographypage:532020/11/24Cryptographypage:54GetURIPUT/etc/issueget,put,

mget,

mputsimple

protocol,http:

textftp:

text,

binary不互相通信的分布式2020/11/24Cryptographypage:55consistent

hash2020/11/24Cryptographypage:562020/11/24Cryptographypage:57HAProxynginx

→

tenginea.jpg2020/11/24Cryptographypage:59web

monitor655352020/11/24Cryptographypage:60url:

提高緩存2020/11/24Cryptographypage:612020/11/24Cryptographypage:622020/11/24Cryptographypage:632020/11/24Cryptographypage:64/bbs/X-Forward-For2020/11/24Cryptographypage:652020/11/24Cryptographypage:66varnishExpire:2013-05-21

14:59:30publicprivate2020/11/24Cryptographypage:68Cache-Control:

max-age=600If-Modify-Since:2020/11/24Cryptographypage:69CDN:

Content

DeliveryNetwork1inCNAMEinCNAME2020/11/24Cryptographypage:702020/11/24Cryptographypage:71Bind,viewBind-dlz

+

MySQL

Pgsql,

Oracle,

db4Squid:

varnishHttpd:

nginxNginx

+

varnishNginx

+

SquidWeb

Cache的類型瀏覽器Cache私有緩存

可以緩存“private”響應(yīng)Cache

ProxySurrogates-緩存Web加速共享緩存

只能緩存“public”響應(yīng)2020/11/24Cryptographypage:72一些數(shù)據(jù)多至43%的Web請(qǐng)求 不可緩存的內(nèi)容Web緩存

為40%左右是比較現(xiàn)實(shí)的2020/11/24Cryptographypage:73頁(yè)面靜態(tài)化理想狀態(tài)：靜態(tài)化所有頁(yè)面——實(shí)際做不到如何靜態(tài)化？由模板生成靜態(tài)頁(yè)面定時(shí)或有更新時(shí)：成千上萬(wàn)個(gè)頁(yè)面的靜態(tài)化不現(xiàn)實(shí)頁(yè)面有變體，不適合靜態(tài)化多臺(tái)服務(wù)器 麻煩頁(yè)面既包含靜態(tài)內(nèi)容，又包含動(dòng)態(tài)內(nèi)容2020/11/24Cryptographypage:74頁(yè)面緩存動(dòng)態(tài)頁(yè)面內(nèi)容按需靜態(tài)化前端cachemod_cacheSquidVarnish*遵循HTTP的Cache規(guī)范可精細(xì)控制CacheExpiresCache-control適合GET類CGI請(qǐng)求2020/11/24Cryptographypage:75HTTP的Cache規(guī)范要充分發(fā)揮Cache的效用，就必須了解HTTPCache規(guī)范和機(jī)制條件請(qǐng)求ExpirationCache-control2020/11/24Cryptographypage:76條件請(qǐng)求2020/11/24Cryptographypage:77If-Modified-Since/Last-Modified1.服務(wù)器響應(yīng)：

200OK

Last-Modified:

…2.瀏覽器請(qǐng)求

GET

…

HTTP/1.1

If-Modified-Since:

…3.服務(wù)器響應(yīng)

200

OK

或

304

Not

Modified條件請(qǐng)求(2)2020/11/24Cryptographypage:78If-None-Match

/

ETag1.Server響應(yīng)

200OK

ETag:

abcdef2.瀏覽器請(qǐng)求

GET

…

HTTP/1.1

If-None-Match:

abcdef3.Server響應(yīng)

200

OK

或

304

Not

ModifiedExpirationExpires設(shè)置對(duì)象的絕對(duì)失效時(shí)間是HTTP/1.0規(guī)范max-age是Cache-control的一個(gè)指令設(shè)置對(duì)象的 （秒數(shù)）是HTTP/1.1規(guī)范max-age優(yōu)先于Expires2020/11/24Cryptographypage:79Cache-controlno-cache=[Set-

]瀏覽器和squid都不可以緩存再次使用前需發(fā)送條件請(qǐng)求max-age=NNN指定緩存有效時(shí)間（秒）public瀏覽器和squid都可以緩存max-age指定緩存時(shí)間private只有瀏覽器可以緩存2020/11/24Cryptographypage:80s,=day-traderHTTP請(qǐng)求攜帶

s:

:

name=profile,domaipath=mysecrets.html,HTTP響應(yīng)設(shè)置

s:Set- :name=profile,domaipath=mysecrets.html,

=day-trader,s——太理想化Cached

content需要的時(shí)候才設(shè)置Squid添加一個(gè)IMS頭2020/11/24Cryptographypage:81一些典型的Cache場(chǎng)景2020/11/24Cryptographypage:82返回個(gè)人信息Cache-control:

private,

max-age=NNNCGI寫操作Cache-control:

no-store,

no-cacheCGI獲取狀態(tài)等信息Cache-control:

public,

max-age=NNNrecvpipeerrorerror2020/11/24Cryptographypage:83passpasslookuplookupGET,

HEADVia:

Expire:

2012-12-21

00:00:00ETag:If-Modified-Since:Last-Modified：If-None-Match2020/11/24Cryptographypage:84reverse

proxy2020/11/24Cryptographypage:85apache,

mod_proxy,

cache

(disk,

memory)Nginx,

cache

(disk,

)squidvarnish

(disk,

memory)6002020/11/24Cryptographypage:862020/11/24Cryptographypage:87Cache-Control:

max-age=10s-maxage=10Varnish

Architecture2020/11/24Cryptographypage:882020/11/24Cryptographypage:89Varnish2020/11/24Cryptographypage:902020/11/24Cryptographypage:91rpmsubroutinesVCL2020/11/24Cryptographypage:922020/11/24Cryptographypage:93Threading

mode2020/11/24Cryptographypage:94Threading

parameters2020/11/24Cryptographypage:95LAMPLAMP2020/11/24Cryptographypage: