Poli Bou » Poli

24 fps on Canon 5D Mark II

Poli — Thu, 18 Mar 2010 22:22:50 +0000

I haven’t been keeping up with news on the 5D Mark II. It was a nice surprise to see Canon release a firmware update to enable 24 fps video in both PAL and NTSC. I’m sure cinematographers everywhere are jumping in their seats. Many people said it couldn’t be done through a firmware update. I bet those were the same people who said manual control was also impossible through a firmware change. In any event, I’m sure everyone is glad to be wrong this time. Cheers to Canon for continuing to update their products.

The new firmware was actually released earlier this week but there was a bug with audio recording in the creative modes. Today, they did a minor fix with the release of firmware version v2.0.4.

Here are the changes:

The Version 2.0.4 firmware includes the movie function enhancements listed in items 1 through 5 below, and also incorporates the correction listed in item 6.

Adds or changes the following movie frame rates.

NTSC:
1920×1080 : 30 fps (changed – actual 29.97 fps)
1920×1080 : 24 fps (added – actual 23.976 fps)
640×480 : 30 fps (changed – actual 29.97 fps)
PAL:
1920×1080 : 25 fps (added – actual 25.0 fps)
1920×1080 : 24 fps (added – actual 23.976 fps)
640×480 : 25 fps (added – actual 25.0 fps)

Adds a function for manually adjusting the sound recording level (64 levels).
Adds a histogram display (brightness or RGB) for shooting movies in manual exposure.
Adds shutter-priority AE mode (Tv) and aperture-priority AE (Av) mode to the exposure modes for shooting movies.
Changes the audio sampling frequency from 44.1 KHz to 48 KHz.
Fixes a phenomenon where communication between the camera and the attached lens is sometimes interrupted after manual sensor cleaning. (This phenomenon only affects units with Firmware Version 1.2.4.)

How to update your firmware:
Updating your firmware is really simple. However, I suggest you follow the official procedure if you have never done it before.

Here’s a simple step by step recap.
Step 1. Format your compact flash card in the 5D mark II.
Step 2. Remove the CF card stick it in your computer’s memory card reader.
Step 3. Open up the CF card in your browser. You will see 2 folders.
Step 4. Download the new firmware. Extract it and copy to the root directory of your CF Card.
Step 5. Place the CF card back in the 5D.
Step 6. Make sure your battery is charged. Important!
Step 7. Navigate to firmware and hit set.
Step 8. The camera should find the new firmware and give you the option to update it.
Step 9. Hit OK, and then don’t touch any buttons on the camera. You will see a progress bar as it goes through the update process which should only take a minute.
Step 10. Format your card again once it has completed. Happy Shooting!

Simple and Clean Memcached

Poli — Fri, 26 Feb 2010 22:54:27 +0000

I just recently moved my company’s web infrastructure out from our colo facility and onto Amazon Web Services (mainly EC2). Hopefully I’ll have a writeup on the whole process soon. We started running a couple of heavy ad campaigns and it severely loaded the servers. At the time I had our Memcached setup disabled while debugging the kinks out from the migration.

With the large influx of traffic it was time to setup Memcached again. This gave me a chance to re-examine how it was done. Previously, we would cache various blocks of a given page. Depending on the situation, this can be a good solution if you needed a combination of cached content and dynamic output. However, this method is a little dirty since you have to modify your existing source code.

Here’s a solution that I’ve come up with. Basically, you scrape the entire HTML output and store that in Memcache. I think its very clean and non-invasive. You don’t have to modify the existing code at all.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54

////////////////////////////////////////////////////////////////
// index.php
//
// This is an example of memcaching a full static page
////////////////////////////////////////////////////////////////

// This is the page you want to cache
$url = "http://www.mysite/mydynamicpage";

// This function grabs the HTML of the page
function ScrapePage () {
global $url;

$ch = curl_init(); // initialize curl handle
curl_setopt($ch, CURLOPT_URL,$url); // set url to post to
curl_setopt($ch, CURLOPT_FAILONERROR, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);// allow redirects
curl_setopt($ch, CURLOPT_RETURNTRANSFER,1); // return into a variable
curl_setopt($ch, CURLOPT_TIMEOUT, 3); // times out after 4s
curl_setopt($ch, CURLOPT_POST, 1); // set POST method
curl_setopt($ch, CURLOPT_POSTFIELDS, ""); // add POST fields
$result = curl_exec($ch); // run the whole process
curl_close($ch);
return $result;
}

// This function stores the HTML into memcache and update if its older than 3 mins
function MemCacheFunction($function_name)
{
$memcache = new Memcache;
$memcache->connect('localhost', 11211) or die ("Could not connect");

$memcache_key = md5('sometextkey'.$function_name);
if ( $memcache_result = $memcache->get($memcache_key) )
{
//echo "It Worked!";
return $memcache_result;
}
//echo "Couldn't Find Key: ".$memcache_key;
$ret = '';
$ret .= $function_name();
$memcache->set($memcache_key, $ret, false, 180);

return $ret;
}

// A simple condition determines whether to load page from Memcache or process the dynamic page
if ($_COOKIE["loggedin"] == "yes") {
include ("index_dynamic.php");
} else {
echo MemCacheFunction("ScrapePage");
}
?>

Typically usage would be for heavy traffic pages such as the homepage of a website. You simply rename your index.php page to something like index_dynamic.php and then use the code below as your original index page. In the code you will have to specify the dynamic page you want to cache and at the bottom a condition to whether load from memcache or the actual page. This part is important. For example, you would only want to show the cache for general traffic such as spiders and visitors who are not logged in.

If you need a good breakdown on installing and configuring Memcached, check this out.

ntsysv – Linux Service Manager

Poli — Tue, 02 Feb 2010 22:43:22 +0000

I was configuring some linux boxes today and discovered the ntsysv command. This is a great command to see what services are loaded during startup. It pops up a simple GUI where you can enable and and disable services. Just run it at the terminal.

Canon Freeze Tag Commerical

Poli — Wed, 27 Jan 2010 06:06:53 +0000

Some pretty awesome technical shots in this commercial. Just thought I’d share.

CANON Commercial – Freeze Tag from Saman Keshavarz on Vimeo.

How do you fix a broken video card? You bake it!

Poli — Mon, 25 Jan 2010 07:59:03 +0000

Seriously. Last October my 3 year old EVGA Geforce 8800GTS started producing artifacts in windows and then on POST. A few days shortly after it completely failed to boot into windows. For some reason, probably pure laziness, I never registered the card which disqualified me from taking advantage of its lifetime warranty. Ouch, lesson learned. So out of options and not wanting to drop some serious coin on a new card I turned to Google with skeptic hopes of a solution.

That’s when I stumbled on a comment that said baking your video card can resurrect it. OK, obviously the commenter must be joking. But wait, further searches reveal the same comment popping up again. Hmmm ok, this is curious…let’s see what happens when I type “baking video cards” into Google.

Right up at the top of the results page is a link to [H]ard|Forum where this whole nonsense originated. After going through the mega thread about baking your card, that’s when I realize this is not another internet pop culture joke, but the real deal with a lot of engineering sense behind it.

Basically, the most common cause of electronic device failure are micro fissure cracks in solder joints which results from stress through use and age. Baking a circuit board allows you to liquefy the solder contacts and eliminate these micro fissure. This is actually a commonly practiced technique called solder reflow. So baking a video card in your kitchen oven is just a crude way to use this method.

After a good amount of reading I was thoroughly prepped to do some cooking. I removed the heat sink and fan off the PCB and placed it in the oven with the door open. Then I set the oven to 395F. I left the door open and did not pre heat the oven because I wanted to gradually raise the temperature so as to not “shock” the board and to slowly get rid of any moisture. After about 15 minutes, I felt everything was warmed up enough to start the actual baking. I closed the door and set the timer for 9 minutes. I stayed near by the entire time in case of disaster.

After 9 minutes I shut the oven off and opened the door and let everything cool down for 30 minutes. Then I put everything back together and popped the card back in the case. And then, the moment of truth…I pushed the power button. No sparks. No shorts. Fans are spinning. That’s good. I hear a beep and see my monitor come out of sleep mode with a lovely memory check in progress. This is looking real good. Thirty seconds later I am logged into windows with no signs of artifacts throughout the whole boot process. This is amazing. I fire up Photoshop and a couple of 3D applications and they all run flawlessly. Now this is unbelievable.

It’s been about four months since my video card has been living on borrowed time. It’s been great, but today it started showing failings signs again. The artifacts are back and windows won’t properly load with the drivers. With some culinary experience under my belt now, I nonchalantly re-baked my video card. I am happy to report another success which not only inspired me to do this write up but it would have been impossible otherwise. I wonder how long I can keep this up .

Disassembled 50mm f/1.4

Poli — Tue, 20 Oct 2009 05:32:27 +0000

Ever wonder what a $300 dollar 50mm f/1.4 lens look like disassembled? Sorry for the lack of close up shots and detailed info as I didn’t plan to do a write up. I got the lens for a good deal off Craigslist since the autofocus was shot. It still worked fine manually but I could tell the motor was struggling due to some fine grit or sand. So I attempted to take it apart and see if I could clean it up and lube the gearing. I had to bend some of the guide rails back in place. They look like they were over extended and became deformed. After thoroughly cleaning the pieces I used some Mobile 1 motor oil to coat any moving part. Haha, I know…probably not the best thing to use on a lens, but I didn’t really plan this out and it was all I had. A month later and the autofocus is still working fine so no harm done .

Using Symbolic Links

Poli — Thu, 11 Jun 2009 18:26:20 +0000

I can never seem to remember all these Linux commands. I just had to look up how to create a symbolic link. Here it is for reference:

ln -s /path/of/original/file /path/of/linked/file

This will point a file or directory to another file or directory. Why would you want to do this? Well, on our web server we have multiple websites and they share a directory of common include files. So instead of using the server path to get the files, we can just link the local include directory to the global include directory and access the files relatively. It just makes things cleaner.

Here’s how to unlink a symbolic link. Remember not to use a trailing slash. rm will also work.

unlink /path/of/symbolic/link
rm /path/of/symbolic/link

Kill A Running MySQL Query

Poli — Mon, 01 Jun 2009 22:40:26 +0000

Somehow you ended up with a query that is taking too long to execute and you need to kill it. First, I’d advise against using this in an UPDATE or INSERT command as I don’t know what will happen. Most likely it’ll will just roll back the DB. For a complex SELECT that just got out of hand, this should be safe.

Log into MySQL from your Linux console.
Show your running queries and find the ID with a long run time. I prefer to check this first in mytop to get a real time view and to see the actual query.
Identify the transaction ID.
Kill it.
Exit and you’re done.

Manual Exposure Control on 5D Mark II Video

Poli — Wed, 27 May 2009 10:29:35 +0000

Looks like we’re finally get manual control! Canon will be releasing a new firmware on June 2, 2009. It will be available here: EOS 5D MarkII Firmware Update Version.

Press Release:

LONDON, UK, 27th May 2009: Canon announced today it will release a firmware update for the EOS 5D Mark II allowing users to manually control exposure when shooting video. The new firmware will be available for download from 2 June 2009 on Canon Europe’s support web site.
Following the launch of the EOS 5D Mark II in September 2008, Canon’s Research and Development team has listened closely to customer feedback to develop additions to the camera’s movie recording functionality.

Allowing EOS 5D Mark II owners to achieve even more stunning video results with the camera, the firmware update will include the following manual controls when shooting video:

Full aperture selection
ISO speed: Auto, 100 – 6400 and H1
Shutter speed: 1/30th – 1/4000th second

The EOS 5D Mark II integrates full HD movie capability into a high-end 21.1 Megapixel camera; opening a multitude of new possibilities for photojournalists and news photographers. Since its launch the camera has proved its appeal to professionals working in diverse fields, from studio and wedding to nature and travel. Now, following customer feedback, Canon has improved functionality for professional video users, further unleashing the potential of the EOS 5D Mark II for cinematographers and photographers alike.

Creating High Quality Stock Chart With Flash

Poli — Fri, 20 Mar 2009 11:10:27 +0000

Working at WallSt I’ve delt with numerous data providers for our financial information. Data feeds are not cheap especially when you have to deal with individual exchange fees. Often times when selecting a provider you are limited by their charts and graphs. You can usually get ticker information in XML, but graphical data is usually restricted to modules or API calls for an image.

One solution is to obtain a historical feed and develop a charting system, but this can be expensive and time consuming. So how can we do this cost effectively and quickly?

Thank you Yahoo. Yahoo provides historical quotes in spreadsheet (CSV – comma separated values) format which can be easily parsed. I have no idea how they can provide this freely. But I won’t question it . Looking at this service, I see that it has changed many times and there are / has been various methods for obtaining their CSV data. So that’s a word for caution should anything change in the future.

Step 1. How to obtain historical quotes:

You can get the data by using the URL below.

http://ichart.finance.yahoo.com/table.csv?s=YHOO

This will provide you with all the EOD price of a company since its IPO. The method above uses the CSV output on a webpage. Yahoo has other URLs that output a downloadable files. Those work also and can be parsed just as easily.

If you want to be able to select a specific date range you can do it in this format:

http://ichart.finance.yahoo.com/table.csv?s=YHOO&a=0&b=1&c=2008&d=11&e=31&f=2008&g=d

Here I’m selecting data just for the year 2008 from 01/01/08 – 12/31/08.

Let’s break down the URL string and take a look at each parameter.

s = The requested ticker symbol.
a = The starting month in two digit format. (ie: “00″ = January, “01″ = February, etc.)
b = The starting day. (ie: 1, 2, … 30, 31)
c = The starting year in four digit format (ie: “2000″)
d = The ending month in two digit format. (ie: “00″ = January, “01″ = February, etc.)
e = The ending day. (ie: 1, 2, … 30, 31)
f = The ending year in four digit format (ie: “2000″)
g = The data interval (ie: “d” = daily, “w” = weekly, ‘m’ = monthly. The default is ‘daily’ if you ignore this parameter.)

Step 2. Retrieving and Parsing the Data

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

////////////////////////////////////////////////////////////////
// data.php
//
// This file will dynamiclly output the CSV locally on your
// server to overcome the security restriction in Flash .
////////////////////////////////////////////////////////////////

function ScrapePage ($url) {
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$fileoutput = curl_exec($ch);
curl_close($ch);
return $fileoutput;
}

$symbol = $_REQUEST["symbol"];

$output = ScrapePage("http://ichart.finance.yahoo.com/table.csv?s=".$symbol);
$output = str_replace("Date,Open,High,Low,Close,Volume,Adj Close\n", "", $output);

echo $output;
?>

I’m using a simple curl function to get the page. Fopen, wget or whatever your weapon of choice should work too. Since we only need the data, I’m removing the column header with a str_replace. The graphing method I will use will take the data in this format so there will be no parsing or manipulating invovlved. That’s all we need to do for this part.

Step 3. Graphing the Data

Building a full fledge charting system in Flash is quite a bit of work. Adobe has simplified this with Flex, which has built in charting support. It’s a great way to go if you have the knowledge and resources to build some very customized. But I’m gonna take this a few more steps down easy street and use a pre compiled flash suite to do the charting.

I will be using charts from amCharts.com. Not just because its free and easy, these are some really well made charts and they look really good. They have a variety of chart types.

Their stock suite can be downloaded here: http://www.amcharts.com/download

I will be using the event chart type in this example. I picked this one because it is the most familiar layout for stocks. It has the standard line chart but also includes the ability to add events. Which for a stock chart, you can labels for news, splits, dividends, etc.

From the download we will need the following files:
/amstock/amstock.swf – The flash file that displays the graph.
/amstock/swfobject.js – Standard flash embed file.
/examples/events/amstock/amstock_setting.xml – The settings file with output parameters

data.php – This is the file we created above.

Once you have all the files, place them in the same directory. Now we’ll edit the files.

First we’ll modify the amstock_setting.xml file.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194

////////////////////////////////////////////////////////////////
// settings.php
//
// This file is based amstock_setting.xml (event chart type).
// I've done a bit of editing to clean up the number display
// and added some sample events.
//
// I've also converted it to php so we can dynamiclly pass
// the ticker symbol.
////////////////////////////////////////////////////////////////
?>

echo ''; ?>

>
>0>
>true>
>
>,>
>.>
>,>
>
number="1000">K>
number="1000000">M>
number="1000000000">B>
>
>
>
did="0">
>East Stock>
>ES>
>7f8da9>
>/post-content/flashchart/data.php?symbol= echo $_POST["symbol"]; ?>>
>
>true>
>YYYY-MM-DD>
>,>
>.>
>
>date>
>open>
>high>
>low>
>close>
>volume>
>adj close>
>
>
>
>
>2007-12-25>
>A>
>>
>
>
>2008-01-01>
>B>
>>
>
>
>2008-12-25>
>C>
>>
>
>
>2009-01-01>
>D>
>>
>
>
>second>
>volume>
>2004-05-12>
>pin>
>>
>true>
>12>
>
>
>second>
>volume>
>2009-03-19>
>pin>
>>
>true>
>12>
>
>
>
>
>
cid="first">
>60>
>>
>#CCCCCC>
>100>
>
>
>EEEEEE>
>
>
>
>true>
>
>100>
>
>fac622>
>
>
gid="close">
>
>close>
>high>
>low>
>
>round_outline>
>
key="false" title="false">Price: {close} High: {high} Low: {low}]]>>
key="false" title="false">Price: {close} High: {high} Low: {low}]]>>
>
>
>
>
cid="second">
>30>
>>
>#CCCCCC>
>100>
>
>
>3>
>
>
>
>
>false>
>
>
>
>true>
>
>0>
>
>db4c3c>
>
>
gid="volume">
>column>
>
>volume>
>
>average>
>100>
>0>
>
key="false" title="false">Volume: {close}]]>>
key="false" title="false">Volume: {close}]]>>
>
>
>
>
>
>
>DD month YYYY>
>
>
>false>
>
>
>
type="DD" count="10">10D>
type="MM" count="1">1M>
type="MM" count="3">3M>
selected="true" type="YYYY" count="1">1Y>
type="YTD" count="0">YTD>
type="MAX">MAX>
>
>Zoom:>
>Custom period:>
>
>
>false>
>
>
>close>
>dragger>
>
>true>
>3>
>
>
>

What I did here was cleaned up some of the XML and added some sample events. I’ve also changed the extension to php since we will be dynamically passing the ticker symbol to the data.php file.

Next we put everything together to display the graph:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47

////////////////////////////////////////////////////////////////
// index.php
//
// Displays the graph and takes a requested symbol
////////////////////////////////////////////////////////////////
?>

$symbol = $_REQUEST["symbol"];
if (!$symbol) {
// if no symbol is inputted defaults to yahoo
$symbol = "YHOO";
}
?>

<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<title>Flash Chart</title>
</head>
<body bgcolor="#FFFFFF">

<script type="text/javascript" src="swfobject.js"></script>
<div id="flashcontent">
<strong>You need to upgrade your Flash Player</strong>
</div>

<script type="text/javascript">
// [CDATA[
var so = new SWFObject("amstock.swf", "amstock", "550", "300", "8", "#FFFFFF");
so.addVariable("path", "");
so.addVariable("settings_file", encodeURIComponent("settings.php?symbol="));
so.write("flashcontent");
// ]]>
</script>

<br>

<form method="post" action="">
Enter Symbol: <input name="symbol" type="text" value="" />
<input name="submit" type="submit" value="Go" />
</form>
</body>
</html>

The form passes our ticker symbol to the setting file which will dynamiclly call the data file with the appropriate symbol.

So there you have it…a nice looking flash graph with historical data. I’ve left out some error checking for the bare minimum to get this done. You can use this technique for other series data. There is one hiccup to all this. It lacks intraday data. That’s pretty vital information when dealing with stock price. However its just a matter of obtaining a feed. Unfortunately, I haven’t been able to find a free source for this data. This would surely seal the deal in creating a no cost flash stock chart that would rival Yahoo and Google.