2012年8月21日
[歌词] 希望和悲伤
听不清,你那隐约的耳语
你的心,仿佛忽远又忽近
我的心,猜不透你我的结局
慢慢靠近,缩短你我的距离
依然小心,避免莫名的抗拒
回首往事,你曾开心地鼓励
搜寻记忆,欢笑陪伴着泪水
而泪水,记录了一天又一天
而欢笑,憧憬着一夜又一夜
希望和悲伤只是我的幻觉
在梦里依然期盼你的出现
不要说过去只是美丽的错误
就让一切留待明天去回忆
希望和悲伤不是我的幻觉
在梦里依然看见你的出现
如果说过去只是美丽的错误
就让一切留待明天去回忆
2012年8月8日
The 5S Rules in IT System Design
Here 5S Rules means: Simple, Solid, Strong, Safe, Smart.
Simple
Simple means: 1. the logic is easy to understand; 2. the architecture has weak coupling connections.
According to Modular Design methodology, generally during design, we firstly define the Domain of the system, and then define Modules within system, and then define Function Points within Module.
When wo do Top-Down design, it's easy to follow Simple Rule, but sometimes the Function grows slowly, it extends from a Function Point to a whole system, in that case, we need to reconstruct the system design, and that's painful.
Let's take an example of Manufacturing System.
There's a work station in the workshop, which needs a worker to do assemlby and quality check after that. So strictly speaking, this worker combins 2 roles: Operator and Quality-checker. But since it requires low skill level and long cycle time, so these 2 functions have been combined to 1 single person. Accordingly, the IT system writes 1 single Logic for that.
But as it requires shorter cycle time and higher efficiency, new worker has been recruted, so this workstaton has been extended to 2 workstations shared with 2 persions, 1 for assembly, 1 for quality checking. At that point, IT system will also be required to split the funtion into 2 Function Points.
To reduce development time, we might add some IF THEN ELSE judgement into original logic, and then different role executes different logic block. That means, load the whole logic first, then execute its sub-logic-block.
Another solution is, analyze the request following Simple Rule even during original design, to package Assembly & Quality-check logic into different Logic Block, and then combine the Logic Block during deployment. That means, 1 person will be assigned with 2 roles' logic before change, and 2 persons will be assigned with their own roles with their own Logic Blocks after change. The background codes only covers Logic Block, and frontground will display it according to role-logicblock configuration.
Solid
Solid means: The logic need to be solid and complete.
The completeness of business logic could help to reduce system exception, and increse system availability.
An effective method of executing Solid Rule is to run Black-box test, let users to write test cases, let users to do the acceptance test.
Because developing guys focus on nornal work flow mostly, but users notice more exceptional work flow, and the test for exceptional work flow helps to increase the completeness of business logic.
Strong
Strong means the system needs to be strong, can support multiply sessions, lots of users, huge data. The system needs to reduce down time throughout all kinds of methods.
As for hardware, we may use Cluster, Load Balance and other technology to increse system availability.
As for software, we may use Cache, Read-write seperation, User grouping, Redundancy, Archive, Database Partition, and other technology to reduce computing time.
One simple method of keeping strong is to find the Bottle Neck of the problem. For example, the bottle neck of one system is running one specific report, which may cause long time waiting if several persons are running it in the same time. The resolution is to increase hardware performance to reduce query time, or to optimize query SQL and Table struchture, or to dispatch report with user permission, such as running report in advance during free time.
Safe
Safe means to avoid service interruptions by all means.
The Black Swan Inccidents - unpredictable exceptional inccidents - are difficult to avoid, the resolution is to accumulate problem resolving methods, and deal with exceptions by alert, logging, notification, so the system can continue to serve.
It will be good to desing a configurable Exception-handling Module, which could take exceptions quickly without rewriting core codes.
Smart
Smart means to use smart methods.
Now we have a lot of open source codes, modules, methodology, and a lot of internet appplications might be introduced into Enterprise systems.
For example, Map-Reduce is known as Google's methodology, it has been used in lots of web sites, and actually we could use it to simplize logic blocks with a lot of IF THEN ELSE judgement.
IT系统设计中的5S原则
5S原则:Simple, Solid, Strong, Safe, Smart.
Simple
Simple就是简单原则,这里的简单有两个意思,一个是逻辑简单容易理解,另一个是架构简单耦合性弱。
按照模块化设计的方法,通常我们在设计时,首先界定系统的边界,然后在系统内划分模块,然后在模块内划分功能点。
通常在进行自顶向下设计的时候,Simple原则是很容易遵守的,但是有时功能是在慢慢增强的,从一个功能点成长为一个系统,在后期开发的时候往往会对前期的设计进行重构,这是相当痛苦的。
举一个制造系统的例子。
比如车间有一个工位,要求工人做装配的活,装配完成后做质量检查。严格地说,这个工人集成了两个角色:装配工和质检员,但是由于生产技术难度低,对生产节拍要求也低,因此为了节省人力成本,这两个动作由同一个人完成。而对应的IT系统功能将逻辑也统一起来,以方便用户操作。
但是后来生产节拍要求越来越高,为了提升效率,车间招募了新的工人,一个工位变成了两个工位,一个人的活变成了两个人的活,其中一人专门做装配,另一人专门做质检。这时要求IT系统将功能拆分出来,将一个逻辑变成了两个逻辑。
通常为了减少开发的时间,我们可以在原先的完整逻辑中增加若干个IF THEN ELSE的判断,然后让不同的角色执行不同的逻辑块。也就是说,先加载逻辑,再判断执行。
另一种设计思路是,在第一次开发的初期,按照Simple原则进行分析,将装配的逻辑和质检的逻辑分别封闭成逻辑块,在实施的时候根据配置的情况进行组合。对应的,变更前一个用户执行两个角色的逻辑,变更后两个用户各执行对应角色的逻辑。后台的代码只负责各自独立的逻辑,前台的展现则根据配置的情况对逻辑进行组合。
Solid
Solid就是逻辑要牢固完整。
业务逻辑的完整可以减少系统的异常,从而提升系统的可用性。
执行Solid原则的一个有效方法是黑箱测试,让用户撰写测试场景,让用户进行接收测试。
开发人员往往想到正常的业务流,而用户则熟悉更多异常的业务流,而对异常业务流的测试可以提升逻辑的完整性。
Strong
Strong就是系统要可靠,能够支持多并发性、多用户、大数据量,通过各种方式减少当机时间。
硬件方面,可以通过Cluster、Load Balance等提升系统的可用性。
软件方面,可以通过缓存、读写分离、用户分组、冗余、归档、数据库分区等技术减少计算时间。
保持Strong的一个简单原则是找到并解决瓶颈所在。比如某个系统的瓶颈是跑某个特定的报表,如果几个人在同一个时间段跑此报表,则造成数据库查询长时间等待。解决的办法是从硬件的角度提升数据库的查询效率,或者优先查询语句及表结构,还有一种办法是在用户允许的情况下,对查询请求进行调度,比如提前在空闲的时间段事先在后台执行查询。
Safe
Safe就是安全原则,通过各种办法避免系统服务的中断。
在IT系统中,黑天鹅事件(无法预测的异常事件)是难以避免的,解决的办法是不断累积异常处理措施,每当有异常的时候,通过报错、日志、通知等方式将异常处理,从而避免系统服务的中断。
系统在设计的时候,应该规划一个可配置的异常处理模块,这样可避免在重写核心代码的情况下实现异常的及时处理。
Smart
Smart就是聪明原则。
现在网络上开源的代码、模块、方法论很多,许多互联网的应用也可以借用到企业系统。
比如Map-Reduce是GOOGLE倡导的方法论,实际上许多包括大量IF THEN ELSE的代码都可以利用Map-Reduce进行逻辑的简化。
2012年7月21日
Walking in the Air中文歌词.lrc
2012年7月20日
What IT System can learn from US Transportation System?
I've travelled to US for 2 weeks at May this year, and was deeply impressed by the high effiency of US Transportation System.
It's said that US is A Country On Wheels, and somehow the Transportation System's effiency affects its Labor Productivity, and the Road Network also improves the balance between city and country.
Here from performance point view, I want to list out some learnings from US Transportation System, which might help us to improve IT System.
1. Read/Write Separation
US roads mainly includes Interstate Road, State Road, and Country Road.
For Interstate Road, it's the main road for long driving Trucks, and a wide Buffer Zone is setup between the 2 directions. For State Road, it might not setup Buffer Zone, but a Mark Line is painted for separation. And as for Country Road, it doesn't need Buffer Zone or Mark Line because of its low traffic.
To understand it in IT language, this design might be regarded as Read/Write Separation.
Considering business request, both Read & Write operations are part of normal procedure, they are not special logicically. But if Read & Write happens into the same Database Object, then we must consider impact of concurrency: Write operation might lock the Table, thus makes Read Operation waiting.
According to Object-Oriented Methodology, if both Read & Write operations are for same Object, then after mapping, both opertions should be defined in the same Table, and this mapping is logically simple.
For example, if we need to read Attribute 1 and to write Attribute 2 of Object A, then after mapping, it equals to Read Column 1 of Table A, and to Write Column 2 of Table A. Since these 2 Attributes are defined in the same Table A, then Writing Column 2 will lock the records of Table A, and Reading Column 1 will be held until the lock released(after Commint/Rollback). So this design might impact performance.
Let's take an example of Manufacturing System.
For Object of Work Order, it has some Meta Information such as Scheduled Date, Product Number, Quantity, and it also has Non-Meta information such as Current Manufacturing Station, Current Stock Location.
From begining of Manufacturing, the Meta information will not be changed, it will only be accessed from Read operation, but the frequency might be high, because every station requests to Read it.
While its Current Manufacturing Station and Current Stock Location will be changed a lot of times, so they will be updated every time, and each time it updates, this kind of Write Operation will lock records of Work Order Table, so the Reading of Meta data might be delayed.
So considering performance, we should define Work Order's Meta information into one Table, and define Current Manufacturing Station, Current Stock Location and other frequently changing Attributs into another Table or other Tables, as a result, the Read & Write operation has been separated, so the Write operation will not impact Read operation.
2. Redundancy
There're 3 examples, which might be understood from concept of Redundancy of IT System.
1) Buffer Zone of City Road
Buffer Zone is setup between 2 directions in City Road, and it's getting narrow right before Street Cross, and an extra Lane appears for Left Turn cars waiting.
The advantage of this design is that the Left Turn cars wouldn't affect direct traffic, after all direct traffic impact total traffic mostly, so its priority should be highest.
2) Waiting Zone of small road
Cars driving in the main road will not slow down before Street Cross with Green Light signal, it will keep normal speed, for the main road has the highest driving priority.
As for cars drving in small road, they might be waiting for quite a long time before getting into main road. So the Waiting Zone should be designed with capacity of having quite a lot of cars waiting.
3) STOP Board
While cars are driving at community, they should be stopped at STOP Board literally - even no one in sight, and drivers must make sure it't clear before continues driving.
This act seems like an obvious redundancy, for the rate of cars meeting is not high for most of time, but the law orders drivers to stop, why?
Because even the incident rate is low, but as a typical Black Swan incident, it's very hard to forecast, and it will be the Bottle Neck while it happens.
So following this rule, although people spend more time at road, but the incident rate is dropped down, so it's acceptable.
As for IT System, the Redundancy design is an effective method to improve performance.
Take examples of Database Query performance, usually there're 3 cases might affect performance.
1) Group By Query
According to design method of Database, during Group By Query, it's quickest while the type of Group By Column is Number Type. And the Query time might be hundreds or thousands of times longer - if the type of Group By Column is Char.
So if business requires Group By Query, at first we should add a redundant Column, which Type is Number, then we map the Number with original Char value. So we Query data Group By Number Column, and then we Join this Dataset with mapping Table, to get result finally.
As from Business point view, this extra Number Column is redundant, but it's a Must to improve performance.
2) Full-table scan
Full-table scan is a very common scenario, it consumes a lot of Query time, and the solution is very simple: adding Index.
From business point view, Index is also redundant, but the affection is great.
3) Recursive Query
Recursive Query is Database's weak point, it consumes a lot of time even Database has Built-in Functions for it.
A common example is BOM structure. Usually BOM uses multiple levels of Parent-Child relationship to build structure, so it requires a lot of Recursive Query to get the whold structure, and it's getting much longer while quantity or level increses.
For this kind of Query, a common solution is prepare the data before business required, by querying BOM, and saving it into a Flat Table or a Materialized View, and the business logic could Query from this Flat Table or Materialized View later.
That means, to use redundant time before buniness required, to use redundat data outside business logic, prepare data in advance, so that we can reduce Query time while business happened.
美国公路系统对IT系统的借鉴
本人于今年5月到美国出了两周的公差,在此期间美国公路系统的发达高效给我留下了深刻的印象。
我们常听说美国是车轮上的国家,从某种意义上来说,公路系统的效率直接影响了国民的劳动生产率,而公路网的健全也促进了城乡之间的均衡。
这里我试图从性能的角度出发,记录几点美国公路系统对IT系统的借鉴。
一、读写分离
美国公路主要分州际公路、州内公路、乡村公路3个级别,大致对应中国的高速公路、国道、省道。
其中州际公路是卡车物流的要道,公路的两个方向之间有很宽的隔离带。州内公路虽然不一定设置隔离带,但是至少通过划线的方式区分行进方向。而乡村公路由于车流量小,往往只有一个车道,因此就没必要区分和标注了。
用IT的语言解读,这个设计相当于读写分离。
从需求的角度出发,读和写的操作都是正常流程的一部分,从逻辑上来理解并没有什么特殊性。但是如果读和写针对同一个数据库对象,那么我们就必须考虑并发性的影响:写操作造成的锁表会导致读操作的等待。
按照面向对象的方法论,如果读和写针对的是同一个对象,那么对象经过映射之后,所有操作针对的属性都应该放在同一个表中,那么这种映射关系的逻辑是最简洁的。
比如对于对象A,要读属性1,要写属性2,那么经过映射,相当于读表A的字段1,写表A的字段2。由于这两个属性放在同一个表中,那么对字段2的写操作会造成表A中这一行数据的锁保护,一直要等到锁保护解除(COMMIT/ROLLBACK)才能进行此行数据的读操作。因此这种设计可能会对性能造成影响。
举一个生产系统的例子。
比如工单这个对象,有工单的元信息如计划生产日期、产品号、数量等,此外还有工单当前所在制造工位和库位等信息。
一旦进入制造环节,那么工单的元信息就不会有变动,对它的操作只有读操作,但是很频繁,因为每个工位都会读取。
而工单的当前工位和库位会不断地变动,因此会不断地写,而每次的写操作都会造成数据行的锁保护,因此影响元信息数据读取。
因此从性能的角度出发,我们应该将工单的元信息放在一个表里,而将当前工位和库位等不断更新的属性放在另一个或若干个表里,从而实现了读写分离,这样写的操作不会影响读的操作。
二、冗余
美国公路系统中有3个例子,可以参照IT系统中冗余的概念进行理解。
1、城市公路中央隔离带
城市公路的2个走向之间有很宽的隔离带,当道路有转向时,隔离带区域缩小,留出一个车位的面积供左转向车等待时停泊。
这样设计的好处是转向车不会影响直行车,毕竟直行车对总体流量的影响是最大的,因为直行车道的畅通是优先级最高的。
2、小路等待区域
行驶在大路上的车辆,如果是绿灯,即使要经过路口,车辆的速度保持在正常行驶速度,司机不会减速行驶,因为大家都认为主路的行驶优先级高。
那么行驶在小路上的车辆,在驶入大路之前,可能会经过较长时间的等待,以避免和主道车辆争抢。因此小路在接入大路之前,所设计的区域应允许较多车辆的临时停留。
3、STOP牌
车辆行驶在住宅或办公区时,在与人车可能交汇的路口设置STOP停车警示牌,司机必须在此牌前将汽车减速停车(即使路口无人),确认安全后再通过。
这个举措表面上看起来是个明显的冗余,因为很多路口人车交汇的概率并不高,但是法律规定司机必须在此牌前停车,其意义何在呢?
这是因为事故出现的概率尽管很低,但是作为黑天鹅事件,是很难预测的,并且一旦出现事故就会造成堵塞交通的瓶颈。
因此尽管大家都增加了道路停留时间,但是恶性事故出现的概率降低了,因此这个冗余是合理的。
对于IT系统来说,冗余设计通常也是提升性能的有效方法。
以数据库查询性能来说,通常影响数据库查询的有以下3种情况:
1、分组查询
按照数据库的设计原理,当进行分组查询的时候,查询的字段类型为整型时效率是最高的,如果用字符型字段进行分组,则查询消耗时间会达成百上千倍之多。
因此当业务需要进行分组查询的时候,首先我们要在数据库里增加冗余字段,将要分组的字符型数据映射成整型数据,将数据按照整型字段汇总,然后将数据集与映射表关联,最终得到我们需要的业务数据。
从业务的角度出发,这个增加的整型字段就是冗余数据,但是它对提升性能是不可或缺的。
2、全表扫描
全表扫描是一个经常碰到的场景,会消耗大量的查询时间,解决方法很简单,是建立索引。
对于业务来说,索引也是冗余数据,但是效果立竿见影。
3、递归查询
对于递归的处理是数据库的弱项,即使数据库有一些处理递归的内置函数,但是往往还是会产生大量的查询时间。
一个常见的例子是BOM的结构。通常BOM采用多级父-子件关系来建立完整的结构,因此在还原时会有大量的递归查询,随着查询数量和层级的增加,查询时间会变得相当长。
对于此类查询,一个常见的解决办法是提前做数据处理,在业务发生之前,将BOM的结构查询出来,并且以展开的平表格式存储到表或物化视图中,这样业务发生时直接查询平表或物化视图即可。
也将是说,利用业务发生前的冗余时间,利用与业务逻辑无关的冗余数据,提前处理从而减少业务发生时的查询时间。
2012年4月23日
Vienna - by Tom Norris
Slow down you crazy child
You're so ambitious for a juvenile
But then if you're so smart tell me
Why You are still so afraid?
Where's the fire, what's the hurry about?
You better cool it off before you burn it out
You got so much to do and
Only So many hours in a day
But you know that when the truth is told
That you can get what you want Or you an just get old
You're gonna kick off before you even
Get halfway through
When will you realize, Vienna waits for you?
Slow down you're doing fine
You can't be everything you want to be
Before your time
Although it's so romantic on the borderline tonight
Tonight,
Too bad but it's the life you lead
You're so ahead of yourself that you forgot what you need
Though you can see when you're wrong, you know
You can't always see when you're right. You're right.
You got your passion you got your pride
But don't you know only fools are satisfied?
Dream on, but don't imagine they'll all come true
When will you realize, Vienna waits for you?
Slow down you crazy child
Take the phone off the hook and disappear for a while
It's alright you can afford to lose a day or two
When will you realize. Vienna waits for you?
2012年1月5日
Android获得mp3文件信息
public boolean GetMp3Info(String mpFullname)
{
try
{
mpFilename = "";
mpTitle = "";
mpArtist = "";
mpAlbum = "";
mpDuration = "";
mpSize = "0";
String selection = MediaStore.Audio.Media.DATA + " = ?"; // like
//String path="/mnt/sdcard/music";
String[] selectionArgs = {"/mnt" + mpFullname};
//String selection = MediaStore.Audio.Media.IS_MUSIC + "!=0";
String [] projection = {
//MediaStore.Audio.Media._ID,
MediaStore.Audio.Media.TITLE,
MediaStore.Audio.Media.ARTIST,
MediaStore.Audio.Media.ALBUM,
MediaStore.Audio.Media.DURATION,
MediaStore.Audio.Media.SIZE
//MediaStore.Audio.Media.DATA, // --> Location
//MediaStore.Audio.Media.DISPLAY_NAME,
};
Cursor cursor = null;
cursor = managedQuery(MediaStore.Audio.Media.EXTERNAL_CONTENT_URI, projection, selection, selectionArgs, null);
// Cursor cursor2 = query(MediaStore.Audio.Media.EXTERNAL_CONTENT_URI, null,
// null, null, MediaStore.Audio.Media.DEFAULT_SORT_ORDER)
mpTitle = cursor.getString(0).toString();
mpArtist = cursor.getString(1).toString();
mpAlbum = cursor.getString(2).toString();
mpDuration = cursor.getString(3).toString();
mpSize = cursor.getString(4).toString();
//cursor.getString(5).toString();
return true;
}
catch (Exception e)
{
return false;
}
} // GetMp3Info() ends