Tuesday, September 04, 2007

Firefox Caching

Federico posted about some work he was doing on making Firefox not cache as many uncompressed bitmaps in memory. I was playing around with the cache stuff and noticed something: my Firefox cache is full of youtube videos. YouTube videos aren't exactly the best thing for Firefox to cache. My internet connection is fast enough that streaming the videos works just fine. I suspect that most people who use online video frequently do so on a connection that can support streaming (otherwise, YouTube would be painfully slow, and they'd go do something else).

It turns out that Firefox's cache is based only on least-recently-used. So, let's say you have a 50 MB cache. Right now, all 50 MB of it is full with cached javascript, css, images, etc. You go to youtube and start watching a 10 MB movie. 20% of your cache gets blown away. In all likelihood, you'll never view that video again.

Even worse is if you listen to a flash-based media player. The MP3s that this downloads are cached just like anything else. So if you listen to 50 MB worth of music your disk cache gets blown away.

Probably LRU isn't the best technique to use here. I'm not sure how one would evaluate various choices (what is a representative test set of browsing sessions?)


Anonymous said...

Well, maybe you could create a "large stuff" cache on the side. Anything that goes over, say, a meg or two gets moved into the large stuff cache.

There are times when, for example, I need to reload a page because I need to enable Javascript, or something crashes and FF needs to be restarted. It'd suck to have downloaded 50 MB worth of video, and then have to start all over again.

Just a random thought.

Ben Maurer said...

Yeah, having a large stuff cache (or, for the same effect -- limiting the amount of cache large objects use) was a solution I thought of.

When making eviction decisions I think frequency should be taken into account somehow.

What's really needed is a good framework for evaluating eviction policies.

Anonymous said...

That explains why Firefox is faster if i run squid on the same computer. Seriously.

Anonymous said...

YouTube doesn't stream. It's progressive download.

quark said...

Yea, Firefox should indeed have a new caching algorithm. The one in use now isn't nearly intelligent enough. I think a good strategy would be to weigh content after a Bayesian algorithm where certain properties makes a file last longer in the cache while other properties makes it last shorter.

Small files should be weighed higher than large files, for example. And certain file types (like JavaScript, CSS and images with a seemingly static URI) should weigh higher than Flash, video and audio files.

To just split the cache in two is in my opinion not enough. The two caches would still not be intelligent enough about what it preserves and what it purges. At least not when the default size for the "small cache" is 50MB: My guess is that this one gets purged rather often.

Anonymous said...

How about using the mime types reported by web server? Afaik for instance flash videos, downloaded mp3 and other files have that distinction from the small and actually useful ones (html, images, css).

All the sane servers should report them correctly. In case not, the browsing continues but with slightly degraded performance at some minor situations.

I wouldn't really use the size alone, as there are huge images on pages (that I really might want to cache), sometimes even huge html, ...

Anonymous said...

Do you think it's firefox or the flash plugin which set it into the tmp dir ?

From my point of view, it's the current implementation of the flash plugin...

mariuz said...

on my system i enabled the cache to be >1G
so i have fast flash loading and caching also for all the js and images
Next would be to install an local proxy with more than 20G cache
My hdd bandwidth is greater than my isp bandwidth

Michael Wolf said...

I don't think your assumption about how long people are willing to wait for videos holds.

Dumb youtube users (there's no lack of them -- read a few videos' comments if you can stomach it) don't realize the value of their time.

The smart ones open a video, hit play, hit pause immediately after, do something else in another tab or program entirely while the video loads, and come back to it later.

Ben Maurer said...

Michael, I don't mind the video being cached while it's being played. Maybe there should be a different accounting category for "items that are being watched right now". But if you have 10 youtube videos open, that shouldn't completely blow away your cache.

Ben Maurer said...

I like the ideas about using the mime type.

On websites, javascript and CSS are loaded synchronously while images are loaded asynchronously. In a sense the javascript and CSS are much more important to cache because they cause latency problems.

Further, HTML documents are much more likely to change than static resources.

Again, what we really need is a way to analyize various eviction policies. There are lots of good ideas, but in order to get a patch in (and to make improvements), it's important to measure what is being done.

Stuart said...

Image data is stored in a separate cache from the file cache fwiw.

Ben Maurer said...

Well, it seems like image data is stored both in an in memory cache *and* in the on disk cache. The in-memory cache is for putting the image on the screen, and stores the image uncompressed.

Anonymous said...

代孕 淘宝刷信用
北京发票 代开发票
餐饮发票 住宿发票
广告发票 对讲机
传世私服 传奇世界私服
新开传世私服 传奇私服
天龙八部私服 天龙私服
手机窃听器 手机窃听器
代写论文 代写论文
北京办证 办证
代孕 代孕网
代孕 代孕
代孕 试管婴儿
代写论文 代写论文
代写代发 论文代写 dhl

modern abstract art sofa manufacturer
净水器 开水器 净水机 净水 软水机 软水 直饮机 家用净水 家用净水器 家用净水机 中央净水 中央净水器 水家装 水家电 水卫士 混合机
过滤机 DHL快递 俄罗斯签证
保险箱 法兰 法兰标准
polycarbonate sheet 回流焊 波峰焊
免烧砖机 注册上海公司 儿童摄影
牛皮癣 皮肤病 制氮机
食堂售餐机 校园一卡通
学校一卡通 ic卡售饭机
食堂售饭机 深圳一卡通
广东售饭机 机电设备安装
北京发票 代开发票
餐饮发票 住宿发票
网络电话 免费网络电话
假发 补发
织发 植发
压滤机 板框压滤机
蒸馏水机 纯蒸气发生器
上海搬家公司 上海搬场公司
大众搬家 大众搬场
张家界旅游 香港旅游
深圳旅行社 打包机
收缩机 对讲机 电源模块
售饭机 水控机 水控器
萎缩性胃炎 neoprene laptop bags
SEO优化 计量泵
胃炎 胃病
冷水机 冰水机
北京特价机票 北京打折计票 北京国际机票
北京机票预定 北京飞机票
北京订机票 北京机票查询 饮料机械
血糖仪 血糖仪
银杏 水培花卉 企业宣传片 空分设备
化工泵 离心机
电话交换机 程控交换机 集团电话 集装袋
混合机 混合机
混合机捏合机 捏合机
捏合机导热油炉 导热油炉
导热油炉 反应釜 反应釜
反应釜 spherical roller bearing
搬运车 搬运车 电动搬运车 油桶搬运车 堆高车 电动堆高车 半电动堆高车 堆垛车
高空作业平台车 电动叉车 平衡重叉车 前移叉车 电瓶叉车
韩国饰品批发 模块电源
X架 超薄灯箱> 易拉宝 展柜制作
代理服务器 游戏加速器 网络加速器
网通加速器 电信加速器 电信网通转换器
电信网通加速器 网通电信互转
网通电信互通 网络游戏加速器
美国VPN代理 美国独享VPN 美国独享IP
pvc ceiling panel Spherical roller bearings
安全鞋 劳保鞋 防砸鞋 电绝缘鞋 上海安全鞋 上海劳保鞋 江苏劳保鞋
服装软件 服装管理软件 进销存软件
进销存管理软件 服装管理系统 服装进销存软件
进销存系统 进销存管理系统 免费进销存软件
吉林中医 东北特产
阳痿 阴茎短小 阴茎增大
早泄 前列腺炎 阴茎增粗 阴茎延长
国际机票 上海国际机票
国际特价机票 国际打折机票
CRM 客户管理软件 客户关系管理
免费客户管理软件 客户管理软件下载 客户信息管理系统 销售管理系统 销售管理
CRM系统 CRM软件 客户关系管理系统
客户关系管理软件 客户管理 客户管理系统 营销管理系统 客户资源管理 销售管理软件 客户资料管理软件 客户资源管理软件
客户信息管理软件 客户资料管理 客户资源管理 客户信息管理 客户资料管理系统
客户资源管理系统 客户管理软件免费版
砂磨机 砂磨机
砂磨机 卧式砂磨机
卧式砂磨机 卧式砂磨机
三辊研磨机 三辊研磨机
三辊研磨机 混合机 混合机
混合机 锥形混合机 锥形混合机 锥形混合机 行星动力混合机 行星动力混合机 行星动力混合机 无重力混合机 无重力混合机 无重力混合机
干粉砂浆设备 干粉砂浆设备
干粉砂浆设备 捏合机 捏合机 捏合机 导热油炉 导热油炉 导热油炉 反应釜 反应釜 反应釜 搪玻璃反应釜 搪玻璃反应釜 搪玻璃反应釜
乳化机 涂料设备 干混砂浆设备 无重力混合机 胶体磨 涂料成套设备 双螺旋混合机
北京婚庆 北京婚庆公司
办证 呼吸机 制氧机
亚都 亚都加湿器 亚都净化器
饰品批发 小饰品批发 韩国饰品 韩国饰品批发 premature ejaculation penis enlargement
破碎机 制砂机 球磨机 雷蒙磨 雷蒙磨粉机 鄂式破碎机 鄂式破碎机 免烧砖机 加气混凝土设备
反击式破碎机 选矿设备
安利产品 马来西亚留学
网站优化 网站推广
论文代写 代写论文
拖链 防护罩 排屑机 塑料拖链 钢铝拖链
深圳装饰 深圳装饰公司 深圳装修公司
特价机票 打折机票 国际机票
新风换气机 换气机 立式新风换气机 风机箱 新风系统 能量回收机
搅拌机 混合机 乳化机
毛刷 毛刷辊 工业毛刷 刷子 钢丝刷
涂层测厚仪 硬度计
兆欧表 激光测距仪
测振仪 转速表
温湿度计 风速仪
噪音计 红外测温仪
硬度计 万用表
美容院 美容加盟
澳洲留学 澳大利亚留学
酒店预定 北京酒店预定 北京酒店
nail equipment nail products nail product nail uv lamp nail uv lamp nail uv lamps uv nail lamp nail brush
nail file nail tool nail tip nail gel curing uv lamps lights
万用表 风速仪
红外测温仪 噪音计
苗木价格 苗木信息 标牌制作 深圳标牌 北京儿童摄影 防静电鞋 淘宝刷信誉
威海凤凰湖 威海海景房 大庆密封件
打标机 淘宝刷信誉 TESOL/TEFL国际英语教师证书 英语教师进修及培训 北京快递公司 北京国际快递