infobright

一 简介

infobright 是基于mysql二次开发的数据仓库,目前社区版免费

二 缺点

不过只能使用“LOAD DATA INFILE”的方式导入数据,不支持INSERT、UPDATE、DELETE, 不支持高并发:只能支持10多个并发查询

三 安装

四 简单实用

五 问题

mogodb 时间日期问题

使用mogonDB遇到了时间日期的问题,之前的sql查询是”select * from table_name where ts between A AND B”

当然mongoDB也有类似的查询办法:
var start = new Date(2011, 3, 1);
var end = new Date(2011, 4, 1);
db.posts.find({ts: {$gte: start, $lt: end}});

而实际去查询的时候,结果总是空的,发现在mongodb里边并未正确的存储为datetime格式
“ts” : “2011-04-19 00:01:00” 这个应该是import的时候当字符存储了,自然没有办法使用日期的范围查询了
正确的格式应该是: “ts” : ISODate(“2010-04-30T16:00:00Z”)
mongodb的import 文档告诉我们: 可以在insert的时候设置{“$date” : 1285679232000} 设置json格式

实际上,使用date格式会非常缓慢,
http://search.cpan.org/~kristina/MongoDB-0.37/lib/MongoDB/DataTypes.pod#Dates
可以看到:”Warning: creating DateTime objects is extremely slow. Consider saving dates as numbers and converting the numbers to DateTimes when needed. A single DateTime field can make deserialization up to 10 times slower.”

因此,把数据转换成时间戳会是比较明智的选择

思科交换机查看状态

#sh ip interface brief
Interface IP-Address OK? Method Status Protocol
FastEthernet0/1 unassigned YES unset down down
FastEthernet0/2 unassigned YES unset up up
能看到端口状态,主要是链路状态

#sh interfaces status
Port Name Status Vlan Duplex Speed Type
Fa0/1 “Connect to Cicso notconnect 8 auto auto 10/100BaseTX
Fa0/2 “Connect to Cicso connected 8 a-full a-100 10/100BaseTX
侧重于vlan和描述性的东西,这里能看到链接速度

nginx php 缓存

这边一直有个php cache的应用,原理是发起一个对自身的请求,保存成静态文件
之前这个应用在apache worker模式跑得很好,换到nginx的fast-cgi 后一直不是很正常

这个问题跟spawn-fcgi的工作原理有关: spawn-fcgi起N个进程,然后FIFO排队处理请求
当有并发>N个PHP CACHE的应用请求过来的时候,php cache的应用会再对自身发起请求,这个请求排在了这些请求的后边,而实际上不会有新的进程来处理这些请求,从而形成死锁,日志中会充斥”upstream timed out” “no live upstreams”

因此,工作模式的排队理论本身决定了这个故障不可避免

解决办法:使用nginx proxy_cache 绕开这个问题,把php cache这个功能交给proxy_cache来完成,nginx的work process遇到阻塞等待的情况,会把这个请求的工作sleep一段时间处理接下来的请求,这个特点明显优于spawn-fcgi的行为

1.建立 /cache/打头的可缓存php location

location ~* ^/cache/(.*\.php)${

}

2. 设置相关的proxy_cache:

1) CACHEZONE:

proxy_cache_path /dev/shm/cache/app.gd.sohu.com levels=1:2 keys_zone=default:100m max_size=4g inactive=20m;

2) CACHE目录:

proxy_read_timeout 5s;
proxy_connect_timeout 5s;
proxy_set_header Host $host;
proxy_cache_use_stale updating;
proxy_cache_key “$host$uri$is_args$args”;
proxy_cache default;

proxy_ignore_headers “Cache-Control”;
proxy_hide_header “Cache-Control”;

proxy_ignore_headers “Expires”;
proxy_hide_header “Expires”;

proxy_hide_header “Set-Cookie”;
proxy_ignore_headers “Set-Cookie”;

add_header Cache-Control max-age=60;

proxy_cache_valid 200 3m;
proxy_cache_valid any 0m;
proxy_temp_path /dev/shm/cache/tmp;

proxy_pass http://serverIP/$1$is_args$args;

3) PURGE设置:

location ~* ^/purge/(.*\.php)$ {
access_log logs/purge_app_access.log sohu;

allow 10.10.0.0/16;
deny all;
set $purge_key “$host/$1$is_args$args”;
proxy_cache_purge default $purge_key;

}

3.需要注意的地方:

1) 影响动态应用能否cache的有几个header:一般是”X-Accel-Redirect”, “X-Accel-Expires”, “Expires” or “Cache-Control” ,跟某些文档提到的cookie是没有关系的,我这里根据自己的应用ignore了 “Expires” 和”Cache-Control” 的header
2) 需要注意,php使用cookie(session也是cookie header实现的一种)来识别用户,如果是私密应用,需要把cookie加入cachekey里边,开放式应用则不必如此
3)设置完毕后可以在访问路径的前边加上/cache/实现php内容的缓存,再加上/purge/就能清除其缓存
4) 需要特别说明,我这边的版本是0.7系列,从测试看1.x系列是会forward cookie的,所以需要抹掉:

proxy_hide_header Set-Cookie;
proxy_ignore_headers Set-Cookie;

2022.07补充

By default, NGINX respects other directives in the Cache-Control header: it does not cache responses when the header includes the PrivateNo-Cache, or No-Store directive. It also doesn’t cache responses with the Set-Cookie header. Further, it only caches responses to GET and HEAD requests. You can override these defaults as described in the answers below.

windows media server 2008 Limit user rights

Microsoft tell us,you can limit user rights in WMS by:

By default, Windows Media Services can only be accessed by users in the Administrators group. To limit user rights effectively while still enabling users to administer Windows Media Services, you can remove users from the Administrators group and then add them to Windows Media Services only. To add users directly to Windows Media Services, you must use Component Services to configure access permissions for the Windows Media Services Component Object Model (COM) object.

To provide a user administrative rights to Windows Media Services
On your server, start the DCOM config utility.

Locate Windows Media Services in the list, and open its properties.

On the Security tab, edit the access permissions.

The Access Permissions list shows the users and user groups that can administer Windows Media Services.

Add the users or groups that you want to be able to administer Windows Media Services, and then close the dialog boxes.

The new settings will take effect when you restart Windows Media Services.

微软告诉我们,可以限制用户权限,使用非管理员的帐号:

默认情况下,只有 Administrators 组中的用户才可以访问 Windows Media Services。若要有效地限制用户权限,同时仍允许用户管理 Windows Media Services,您可以从 Administrators 组中删除用户,然后将其仅添加到 Windows Media Services。若要将用户直接添加到 Windows Media Services,您必须使用“组件服务”为 Windows Media Services 组件对象模型 (COM) 对象配置访问权限。
为用户提供对 Windows Media Services 的管理权限的步骤
在您的服务器上,启动 DCOM 配置实用工具。

在列表中找到 Windows Media Services,并打开其属性。

在“安全”选项卡上,编辑访问权限。

“访问权限”列表显示了可以管理 Windows Media Services 的用户和用户组。

添加希望能够管理 Windows Media Services 的用户或组,然后关闭对话框。

在重新启动 Windows Media Services 后,新的设置将生效。

In fact,when you open DCOMcnfg,you will find that the Security Permission of “Windows Media Services” is grey and disabled.

实际上,当你打开组件管理器,你会发现”Windows Media Services”的权限设置选项是灰色的,无法修改.

This is the new security feather of win2008 R2 x64. Some core system components only grant the local internal account, TrustedInstaller, Full Control permission instead of the local Administrators group.

这个是win2008 R2 x64的安全特性.一些核心系统组件只能允许本地帐号,TrustedInstaller有完全控制权限,而不是本地管理员组.

To be able to modify the settings of “Windows Media Services” on a Windows Server 2008 R2 system, you need to grant the local Administrators group permissions to its registry key as follows:

要在Windows Server 2008 R2 上修改 “Windows Media Services” ,你需要授予本地管理员组有注册表权限:

1. Check the Appid of WMS in “Component Services management console (dcomcnfg.exe)”.On my machine it is {A2EFA5CB-3B0E-11D2-9EFD-006097D2D7CF}
2. Run Regedit.exe and browse to “HKEY_CLASSES_ROOT\AppID\{A2EFA5CB-3B0E-11D2-9EFD-006097D2D7CF}” key.
3. Secondary-mouse click on the {A2EFA5CB-3B0E-11D2-9EFD-006097D2D7CF} key and select Permissions…
4. Click the Advanced button in the Permissions window and select the Owner tab. Under Change owner to select the local Administrators group and click on Apply, then OK.
5. Then under Permissions window, select the local Administrators group and under Permissions for Administrators select Full Control and click on Apply, then OK.
NOTE: DO NOT modify/change any permissions for the TrustedInstaller account.

1.在组件管理器中查到WMS的appid,在我的机器上,它的ID是{A2EFA5CB-3B0E-11D2-9EFD-006097D2D7CF}
2.使用regedit,定位到HKEY_CLASSES_ROOT\AppID\{A2EFA5CB-3B0E-11D2-9EFD-006097D2D7CF}”
3.右键点击{A2EFA5CB-3B0E-11D2-9EFD-006097D2D7CF},选权限
4.在权限设置窗口中点高级,选择所有者标签,选中administrators为所有者
5.授予administrators组有读/写等所有权
注意:务必不要修改TrustedInstaller的权限

Now,re-run Component Services management console (dcomcnfg.exe), you can follow the steps of Microsoft to configure access permissions for the Windows Media Services Component Object Model (COM) object.

现在,重新执行组件管理器,你就能根据前文中微软的步骤来配置Windows Media Services的访问权限了.

PS.There is a bug of Windows Media Services in win2003(not in win2008),if you(limit user) exit the wmsadmin and then start it again, you will get error code 0xc00d0006 and the server no longer shows up in the wmsadmin. You have to once again add it using “localhost” every time .

PS. windows 2003 的 Windows Media Services有个bug(win2008没有):每次退出wmsadmin再次开启,受限用户会收到错误0xc00d0006提示说无法与服务器建立连接,并且服务器不再显示在wmsadmin中,你必须再次添加”localhost”进去.

see more about:

http://technet.microsoft.com/en-us/library/cc753241(WS.10).aspx
http://blogs.msdn.com/b/emeadaxsupport/archive/2010/01/26/unable-to-edit-the-dcom-settings-for-iis-wamreg-admin-service-on-a-windows-server-2008-r2-when-trying-to-configure-kerberos-authentication-for-role-centers.aspx

IBM x3650M3 disk fail

IBM x3650M3 使用的阵列卡 ServeRAID M10XX and M50XX使用了power save技术
这种不成熟的技术将会导致磁盘wake up的过程中出现不可知的系统故障
比如:

sd 0:2:0:0: timing out command, waited 360s
sd 0:2:0:0: SCSI error: return code = 0x06000000
end_request: I/O error, dev sda, sector 528987842
ext3_abort called.
EXT3-fs error (device sda5) in ext3_dirty_inode: Journal has aborted
__journal_remove_journal_head: freeing b_committed_data
unable to read inode block – inode=898882, block=917148
EXT3-fs error (device sda3) in ext3_dirty_inode: Journal has aborted
EXT3-fs error (device sda3) in ext3_dirty_inode: Journal has aborted
ext3_abort called.
EXT3-fs error (device sda3): ext3_journal_start_sb: Detected aborted journal
Remounting filesystem read-only

使用如下命令检测是否启用了节电模式:
> MegaCLI -AdpGetProp DefaultLdPSPolicy -a0
> Sample Output: “Adapter 0: Default power savings policy : Automatic”
以上输出说明节电模式启用,应当关闭

关闭节电模式:
> MegaCLI -AdpSetProp -DefaultLdPSPolicy -None -a0
or
> MegaCLI -LDSetPowerPolicy None -Lall -aALL
> Sample Output: “Adapter 0: Default power savings policy : None”
以上输出说明节电模式已经关闭

检查热备盘(hot spare)的节电策略:
> MegaCLI -AdpGetProp DsblSpinDownHSP -aALL
> Sample Output: “Adapter 0: Disable spin Down of Hot Spares: Disabled”
以上输出说明热备盘(hot spare)的”禁用了spin down模式禁用设置”,也就是启用的
修改热备盘(hot spare)的节电策略:
> MegaCLI -AdpSetProp -DsblSpinDownHSP -val -aALL

相关链接:http://www-947.ibm.com/support/entry/portal/docdisplay?lndocid=MIGR-5087494

iphone QQ 端口

今天有个颇蛋疼的需求,iphoneQQ 在公司不是很正常,初步怀疑是端口没开放导致的
1.查看手机的IP,在cisco 6506上建立策略路由
1)建立ACL
access-list extended 188
permit ip host $iphoneIP$ any

2)建立策略路由

route-map src-route permit 20(原来有这个策略路由,新增个规则)
match ip address 188
set ip next-hop $linuxNAT$

3)在手机的VLAN接口生效

ip policy route-map src-route

2. 在linux设置NAT转发请求:

/sbin/iptables -t nat -A POSTROUTING -s $iphoneIP$/32 -o eth0 -j MASQUERADE
sysctl -w net.ipv4.ip_forward=1

3.在linux NAT机器上听包:

tcpdump -nn host $iphoneIP$

抓到了,iphoneQQ访问的是14000端口