tag:blogger.com,1999:blog-58229462019-05-19T20:09:41.727-05:00Marc's MusingsAll the (developer) news that's unfit to read...IDisposablehttp://www.blogger.com/profile/02275315449689041289noreply@blogger.comBlogger191125tag:blogger.com,1999:blog-5822946.post-2657464125969159272017-03-31T18:43:00.001-05:002019-05-19T20:09:00.566-05:00Codeplex shutting down, moved the repos to GitHub<p>The four simple projects I used to have on CodePlex have been migrated to GitHub.</p><h2><a href="https://github.com/IDisposable/IFilterExtractor">IFilter Extractor</a></h2><p>A text extracting <b>COM</b> component that uses any installed IFilter (Microsoft Indexing) to extract text from files. Information in <a href="/search/label/IFilter">old blog posts</a>.</p><h2><a href="https://github.com/IDisposable/Dynamic">Dynamic Reflection Library</a></h2><p>A library for doing lightweight-code-generation in the era before the expression were a thing. Lots of <a href="/search/label/Dynamic">old blog posts</a> around here about it.</p><h2><a href="https://github.com/IDisposable/URITemplate">URI Template and URI Patterns</a></h2><p>The UriTemplate library is a simple set of code to encapsulate the building and parsing of URIs from replacement tokens. Using this couple of classes will make it easy to build RESTful URIs. Information in <a href="/search/label/UriTemplate">old blog posts</a></p><h2><a href="https://github.com/IDisposable/ASPNetRSSToolkit">ASP.Net RSS Toolkit</a></h2><p>An ASP.Net component set for publishing and consuming RSS/Atom feeds in .Net 2.0. Includes a datasource component. This was one of the first things Microsoft open-sourced (sadly in an era where that really meant mostly abandoned). I took the sources and floated a CodePlex repository and then promptly abandoned it myself. Some information in <a href="/search/label/RssToolkit">old blog posts</a>.</p>IDisposablehttp://www.blogger.com/profile/02275315449689041289noreply@blogger.com0tag:blogger.com,1999:blog-5822946.post-27292868392599513902013-12-03T20:08:00.001-06:002013-12-03T20:09:57.751-06:00EntityFramework 6 breaks Backup/Restore of LocalDB.<a href="http://entityframework.codeplex.com/releases/view/87028" target="_blank">EntityFramework 6</a> has lots of <a href="http://entityframework.codeplex.com/wikipage?title=Connection%20Resiliency%20Spec" target="_blank">Resiliency enhancements</a>, but one of the side effects is that doing a backup or restore of a LocalDB database <a href="http://entityframework.codeplex.com/discussions/454994" target="_blank">will need a tweak</a> to keep EF from spinning up a transaction around the SQL statement.<br />
<br />
Essentially, when you call ExecuteSqlCommand, you have to request that it does NOT ensure the a transaction exists with the first parameter of <b>TransactionalBehavior.DoNotEnsureTransaction</b>.<br />
<br />
I have updated the appropriate posts:<br />
<br />
<a href="http://musingmarc.blogspot.com/2012/11/how-to-backup-localdb-database-under.html" target="_blank">How to backup a LocalDB database under an MVC 4 sytem</a><br />
<a href="http://musingmarc.blogspot.com/2012/11/how-to-backup-localdb-database-under.html" target="_blank">Restoring a LocalDB within an MVC web application</a><br />
<br />
<br />IDisposablehttp://www.blogger.com/profile/02275315449689041289noreply@blogger.com0tag:blogger.com,1999:blog-5822946.post-76640982735591149052013-05-24T22:27:00.002-05:002013-05-30T18:31:03.315-05:00Getting the requestor's IP address.<h1>
Getting the client's IP</h1>
While it seems it should be straightforward to get the IP address of the client making a web request in the ASP.Net pipeline, there are a few surprises lurking. In my link-click tracking application, I ended up with this snippet based on Grant Burton's <a href="http://www.grantburton.com/2008/11/30/fix-for-incorrect-ip-addresses-in-wordpress-comments/">great post</a>.<br />
<div>
<pre>namespace Phydeaux.Helpers
{
using System;
using System.Net;
using System.Net.Sockets;
using System.Web;
public static class ClientIP
{
public static string ClientIPFromRequest(this HttpRequestBase request, bool skipPrivate)
{
foreach (var item in s_HeaderItems)
{
var ipString = request.Headers[item.Key];
if (String.IsNullOrEmpty(ipString))
continue;
if (item.Split)
{
foreach (var ip in ipString.Split(','))
if (ValidIP(ip, skipPrivate))
return ip;
}
else
{
if (ValidIP(ipString, skipPrivate))
return ipString;
}
}
return request.UserHostAddress;
}
private static bool ValidIP(string ip, bool skipPrivate)
{
IPAddress ipAddr;
ip = ip == null ? String.Empty : ip.Trim();
if (0 == ip.Length
|| false == IPAddress.TryParse(ip, out ipAddr)
|| (ipAddr.AddressFamily != AddressFamily.InterNetwork
&& ipAddr.AddressFamily != AddressFamily.InterNetworkV6))
return false;
if (skipPrivate && ipAddr.AddressFamily == AddressFamily.InterNetwork)
{
var addr = IpRange.AddrToUInt64(ipAddr);
foreach (var range in s_PrivateRanges)
{
if (range.Encompasses(addr))
return false;
}
}
return true;
}
/// <summary>
/// Provides a simple class that understands how to parse and
/// compare IP addresses (IPV4 and IPV6) ranges.
/// </summary>
private sealed class IpRange
{
private readonly UInt64 _start;
private readonly UInt64 _end;
public IpRange(string startStr, string endStr)
{
_start = ParseToUInt64(startStr);
_end = ParseToUInt64(endStr);
}
public static UInt64 AddrToUInt64(IPAddress ip)
{
var ipBytes = ip.GetAddressBytes();
UInt64 value = 0;
foreach (var abyte in ipBytes)
{
value <<= 8; // shift
value += abyte;
}
return value;
}
public static UInt64 ParseToUInt64(string ipStr)
{
var ip = IPAddress.Parse(ipStr);
return AddrToUInt64(ip);
}
public bool Encompasses(UInt64 addrValue)
{
return _start <= addrValue && addrValue <= _end;
}
public bool Encompasses(IPAddress addr)
{
var value = AddrToUInt64(addr);
return Encompasses(value);
}
};
private static readonly IpRange[] s_PrivateRanges =
new IpRange[] {
new IpRange("0.0.0.0","2.255.255.255"),
new IpRange("10.0.0.0","10.255.255.255"),
new IpRange("127.0.0.0","127.255.255.255"),
new IpRange("169.254.0.0","169.254.255.255"),
new IpRange("172.16.0.0","172.31.255.255"),
new IpRange("192.0.2.0","192.0.2.255"),
new IpRange("192.168.0.0","192.168.255.255"),
new IpRange("255.255.255.0","255.255.255.255")
};
/// <summary>
/// Describes a header item (key) and if it is expected to be
/// a comma-delimited string
/// </summary>
private sealed class HeaderItem
{
public readonly string Key;
public readonly bool Split;
public HeaderItem(string key, bool split)
{
Key = key;
Split = split;
}
}
// order is in trust/use order top to bottom
private static readonly HeaderItem[] s_HeaderItems =
new HeaderItem[] {
new HeaderItem("HTTP_CLIENT_IP",false),
new HeaderItem("HTTP_X_FORWARDED_FOR",true),
new HeaderItem("HTTP_X_FORWARDED",false),
new HeaderItem("HTTP_X_CLUSTER_CLIENT_IP",false),
new HeaderItem("HTTP_FORWARDED_FOR",false),
new HeaderItem("HTTP_FORWARDED",false),
new HeaderItem("HTTP_VIA",false),
new HeaderItem("REMOTE_ADDR",false)
};
}
}
</pre>
</div>
IDisposablehttp://www.blogger.com/profile/02275315449689041289noreply@blogger.com6tag:blogger.com,1999:blog-5822946.post-72398728400360985912012-12-10T21:08:00.001-06:002013-12-03T20:10:09.282-06:00Restoring a LocalDB within an MVC web application<i>UPDATE: <a href="http://entityframework.codeplex.com/releases/view/87028" target="_blank">EntityFramework 6</a> has lots of <a href="http://entityframework.codeplex.com/wikipage?title=Connection%20Resiliency%20Spec" target="_blank">Resiliency enhancements</a>, but one of the side effects is that this needs a tweak to keep EF from spinning up a transaction around the SQL statement. Essentially, you have to call ExecuteSqlCommand and request that it does NOT ensure the a transaction exists with the first parameter of <b>TransactionalBehavior.DoNotEnsureTransaction</b>. If still running EF 5 or below, omit that argument. </i><br />
<br />
So, as a follow-up to <a href="http://musingmarc.blogspot.com/2012/11/how-to-backup-localdb-database-under.html">backing up a LocalDB database</a>, I guess I should show the simplest path to restoring one.<br />
<br />
So, without further adéu, I give you:<br />
<blockquote>
<pre>public class RestoreDatabaseModel
{
public HttpPostedFileBase File { get; set; }
}
//
// GET: /Admin/RestoreDatabase
[Authorize(Roles = "Admin")]
public ActionResult RestoreDatabase()
{
return View(new RestoreDatabaseModel());
}
//
// POST: /Admin/RestoreDatabase
[Authorize(Roles = "Admin")]
[HttpPost]
public ActionResult RestoreDatabase(RestoreDatabaseModel model)
{
const string YOURAPPNAME = "YourAppName";
var dbPath = Server.MapPath(String.Format("~/App_Data/Restore_{0}_DB_{1:yyyy-MM-dd-HH-mm-ss}.bak", YOURAPPNAME, DateTime.UtcNow));
try
{
model.File.SaveAs(dbPath);
using (var db = new DBContext())
{
var cmd = String.Format(@"
USE [Master];
ALTER DATABASE {0} SET SINGLE_USER WITH ROLLBACK IMMEDIATE;
RESTORE DATABASE {0} FROM DISK='{1}' WITH REPLACE;
ALTER DATABASE {0} SET MULTI_USER;"
, YOURAPPNAME, dbPath);
db.Database.ExecuteSqlCommand(TransactionalBehavior.DoNotEnsureTransaction, cmd);
}
ModelState.AddModelError("", "Restored!");
}
catch (Exception ex)
{
ModelState.AddModelError("", ex);
}
return View(model);
}</pre>
</blockquote>
This iteration saves the posted file based on the current date-time and the supplies the correct commands to restore the database. I leave the flushing of HttpCache to you...<br />
<br />
Also, if you haven't extended the upload limits of a POST, you'll need this (or similar) in your web.config<br />
<blockquote>
<pre><system.web>
<httpRuntime maxRequestLength="40960" targetFramework="4.5">
</httpRuntime>
</system.web></pre>
</blockquote>
This enables larger files to be uploaded (in this case, 40MB... if your database backup is bigger than that, you have no business hacking around with a LocalDB... get a real SQL Server instance to point at.IDisposablehttp://www.blogger.com/profile/02275315449689041289noreply@blogger.com0tag:blogger.com,1999:blog-5822946.post-60582639049115837122012-12-04T16:27:00.000-06:002012-12-04T16:30:25.932-06:00How to create random readable strings for .Net application<h3>
Why would I want to be random?</h3>
If you need a random string, I assume you know why you're here. However there are some common uses for random strings I want to list out for the Google juice factor:<br />
<ol>
<li>CAPTCHA codes when not using something cool like <a href="http://www.google.com/recaptcha" target="_blank" title="Stop Spam, Read Books">reCAPTCHA</a></li>
<li>Email <a href="http://stackoverflow.com/questions/3673574/is-a-random-string-a-good-verification-code" target="_blank" title="Is a random string a good verification code">verification codes</a>.</li>
<li><a href="http://en.wikipedia.org/wiki/Cryptographic_nonce" target="_blank" title="Cryptographic Nonce">Nonce</a> values for challenge/response.</li>
<li><a href="http://en.wikipedia.org/wiki/Salt_(cryptography)" target="_blank" title="Cryptographic Salt">salt</a> values to increase entropy on password hashes.</li>
<li>Registration codes.</li>
</ol>
<h3>
How should I generate them in .Net?</h3>
There is a simple answer, really. In .Net you can just make a call to <a href="http://msdn.microsoft.com/en-us/library/system.security.cryptography.rngcryptoserviceprovider(v=vs.100).aspx" target="_blank" title=" MSDN - RNGCryptoServiceProvider">RNGCryptoServiceProvider</a>'s <a href="http://msdn.microsoft.com/en-us/library/system.security.cryptography.rngcryptoserviceprovider.getnonzerobytes(v=vs.100).aspx" target="_blank" title="MSDN - GetNonZeroBytes"> GetNonZeroBytes</a> method and convert those bytes to characters.<br />
<blockquote>
<pre>var random = new byte[16]; // whatever size you want
var rng = new RNGCryptoServiceProvider();
rng.GetNonZeroBytes(random); // Fill with non-zero random bytes
return Convert.ToBase64String(random); // convert to a string.</pre>
</blockquote>
If you have the <a href="http://nuget.org/packages/Microsoft.AspNet.Mvc" target="_blank">MVC 4</a> package available, you can use the convenient <a href="http://msdn.microsoft.com/en-us/library/system.web.helpers.crypto.generatesalt(v=vs.111).aspx" target="_blank">Crypto.GenerateSal</a>t method as a shorthand as it essentially does the above code.<br />
<br />
This, of course, limits the returned string to the <a href="http://en.wikipedia.org/wiki/Base_64" target="_blank">Base-64</a> characters.<br />
<h3>
When should I care about the contents?</h3>
In general, you don't care about the contents of the random string. The one generated by logic above is pretty useful as it is a wide set of all-ASCII characters that will not get you in trouble when crossing <a href="http://blogs.msdn.com/b/michkap/archive/2005/01/22/358675.aspx" target="_blank">code-pages</a>.<br />
<br />
The biggest downside of this approach is that the string is only using the a 64 character set, so you're excluding a lot of other possible characters, but in most applications that isn't a problem. In fact, quite the opposite is true. In many cases, we might want to avoid specific characters like the <a href="http://en.wikipedia.org/wiki/%2B" style="font-style: italic; font-weight: bold;" target="_blank">+ character</a> because this might be used in a URL In other cases, you might want to generate a fuller character set (or a specific set like an all-<a href="http://en.wikipedia.org/wiki/Emoji" target="_blank">emoji</a> string).<br />
<br />
A more common need, though, would be if you need to put something on screen for a user to type (such as a registration code) that should not be easy to mistake characters. In some fonts, the characters <i>1, l and I</i> or <i>0, o and 0</i> are very easily mistaken. For such cases, you can use a function like the following to generate a reasonably readable string<br />
<blockquote>
<pre>namespace Silly
{
using System.Security.Cryptography;
public static partial class Helpers
{
public static string RandomReadableString(int length)
{
return "23456789ABCDEFGHJKMNPQRSTUVWXYZabcdefghijkmnpqrstuvwxyz".RandomString(length);
}
public static string RandomString(this string characterSet, int length)
{
var rng = new RNGCryptoServiceProvider();
var random = new byte[length];
rng.GetNonZeroBytes(random);
var buffer = new char[length];
var usableChars = characterSet.ToCharArray();
var usableLength = usableChars.Length;
for (int index = 0; index < length; index++)
{
buffer[index] = usableChars[random[index] % usableLength];
}
return new string(buffer);
}
}
}</pre>
</blockquote>
You can call the second function against any string of characters. For example I'm using the <em>RandomReadableString</em> method to generate email confirmation codes that can easily be typed if needed.<br />
<h3>
Boring! Spice it up...</h3>
For even more fun, here's some Emoji sequences that can be used for eye charts or stupid <strike>pet</strike> code tricks.<br />
<blockquote>
// Emoji fun<br />
// random weather "☀☁☂☃"<br />
// random finger pointers "☜☝☞☟"<br />
// random zodiac "♈♉♊♋♌♍♎♏♐♑♒♓"<br />
// random chess pieces "♔♕♖♗♘♙♚♛♜♝♞♟"<br />
// random music notation "♩♪♫♬♭♯"<br />
// random trigrams "☰☱☲☳☴☵☶☷"<br />
// random planets "♃♄♅♆♇"</blockquote>
IDisposablehttp://www.blogger.com/profile/02275315449689041289noreply@blogger.com4tag:blogger.com,1999:blog-5822946.post-75603410252153048552012-11-27T23:53:00.001-06:002013-12-03T20:10:17.204-06:00How to backup a LocalDB database under an MVC 4 sytem<i>UPDATE: <a href="http://entityframework.codeplex.com/releases/view/87028" target="_blank">EntityFramework 6</a> has lots of <a href="http://entityframework.codeplex.com/wikipage?title=Connection%20Resiliency%20Spec" target="_blank">Resiliency enhancements</a>, but one of the side effects is that this needs a tweak to keep EF from spinning up a transaction around the SQL statement. Essentially, you have to call ExecuteSqlCommand and request that it does NOT ensure the a transaction exists with the first parameter of <b>TransactionalBehavior.DoNotEnsureTransaction</b>. If still running EF 5 or below, omit that argument. </i><br />
<br />
I have a smallish MVC 4 site with a database. In this case it isn't worthy of a dedicated SQL Server, so I decided to try out the new LocalDb database feature.<br />
<br />
While the database isn't particularly mission critical, I would like to be able to easily back it up on demand to allow some level of disaster recovery.<br />
<br />
So, without further adéu, I give you:<br />
<br />
<pre>namespace SomeSimpleProject.Controllers
{
[Authorize(Roles="Admin")]
public class BackupController : Controller
{
public ActionResult BackupDatabase()
{
var dbPath = Server.MapPath("~/App_Data/DBBackup.bak");
using (var db = new DbContext())
{
var cmd = String.Format("BACKUP DATABASE {0} TO DISK='{1}' WITH FORMAT, MEDIANAME='DbBackups', MEDIADESCRIPTION='Media set for {0} database';"
, "YourDB", dbPath);
db.Database.ExecuteSqlCommand(TransactionalBehavior.DoNotEnsureTransaction<i style="font-weight: bold;">, </i>cmd);</pre>
<pre> }
return new FilePathResult(dbPath, "application/octet-stream");
}
}
}
</pre>
<br />
<br />IDisposablehttp://www.blogger.com/profile/02275315449689041289noreply@blogger.com3tag:blogger.com,1999:blog-5822946.post-44431575980178342542011-12-15T16:19:00.000-06:002011-12-15T16:19:16.907-06:00Obligatory linkFound this <a href="http://www.hanselman.com/blog/LinkbaitHackerSlashNewsDotConsideredCancerousRequestForCallToAction.aspx">link</a>IDisposablehttp://www.blogger.com/profile/02275315449689041289noreply@blogger.com0tag:blogger.com,1999:blog-5822946.post-54399454978912658242011-01-31T00:42:00.000-06:002011-01-31T00:42:20.229-06:00It's that time of year again... International Zebra Day is upon us.So, as in many years past, I once again am celebrating International Zebra Day. If you want ideas about how to celebrate, check <a href="http://musingmarc.blogspot.com/2007/01/best-to-you-and-yours-on-international.html">here</a> and <a href="http://musingmarc.blogspot.com/2005/01/happy-international-zebra-day.html">here</a> for past posts.
This year, I have to add that two more countries have signed on... perhaps you can spread the striped love.IDisposablehttp://www.blogger.com/profile/02275315449689041289noreply@blogger.com0tag:blogger.com,1999:blog-5822946.post-39577164201145942382010-09-28T17:57:00.000-05:002010-09-28T17:57:02.644-05:00MS10 - 070 Post Mortem analysis of the patchNow that Microsoft <a href="http://weblogs.asp.net/scottgu/archive/2010/09/28/asp-net-security-update-now-available.aspx">has patched</a> the <a href="http://weblogs.asp.net/scottgu/archive/2010/09/24/update-on-asp-net-vulnerability.aspx">POET vulnerability</a>, I thought it would be instructive to see what they changed. Now I'm not masochistic enough to disassemble aspnet_wp.exe or webengine.dll so there are things changed there that I don't know... but I did do a <a href="http://www.red-gate.com/products/reflector/">Reflector</a> Export of the System.Web and System.Web.Extensions assemblies and then used <a href="http://www.scootersoftware.com/moreinfo.php?zz=moreinfo_compare">BeyondCompare</a> to do a diff against the files.
<br />
<h2>
The analysis is simple:</h2>
<ol>
<li>Don't <a href="http://www.owasp.org/index.php/Top_10_2007-Information_Leakage_and_Improper_Error_Handling">leak exception</a> information - This prevents exploits from seeing what is broken.</li>
<li>Don't short-circuit on padding checks (take the same amount of time for padding correct verses padding broken) - This prevents exploits from seeing <a href="http://www.owasp.org/index.php/Covert_timing_channel">timing difference</a> for incorrect padding.</li>
<li>Don't be too picky about exception catching in IHttpHandler.ProcessRequest - This prevents exploits from seeing that you caught one kind of exception (CryptographicException) instead of all exceptions.</li>
<li>Switch from <a href="http://en.wikipedia.org/wiki/Cryptographic_hash_function">Hash</a>-based <a href="http://en.wikipedia.org/wiki/Initialization_vector">initialization vectors</a> to <a href="http://en.wikipedia.org/wiki/Random">Random</a> IVs - This prevents exploits from using the relationship between the data and the hash to decrypt faster.</li>
<li>Allow for backward compatibility - In case this breaks something, allow the new behavior to be <a href="http://www.mskbarticles.com/index.php?kb=2425938">reverted</a> in-part.</li>
<li>When doing a code review pass, change to make it clear you've considered the new options.</li>
</ol>
<h3>
Here's the blow-by-blow review</h3>
<h4>
Changes in System.Web</h4>
<ul>
<li><b>AssemblyInfo.cs</b><br />
(v2.0/v3.5) Bump AssemblyFileVersion from 2.0.50272.5014 to 2.0.50727.5053<br />
(v4.0) Bump AssemblyFileVersion from 4.0.30319.1 to 4.0.30319.206</li>
<li><b>ThisAssembly.cs</b><br />
(v2.0/v3.5) Bump InformationalVersion from 2.0.50272.5014 to 2.0.50727.5053<br />
(v4.0) Bump the BuildRevisionStr from 1 to 206<br />
(v4.) Bump InformationalVersion from 4.0.30319.1 to 4.0.30319.206</li>
<li><b>HttpCapabilitiesEvaluator.cs</b><br />
When caching the browser capabilities, sets the cache expiration to sliding time based on the the setting specified (instead of no sliding, no absolute)</li>
<li><b>MachineKeySection.cs</b><br />
Calls to EncryptOrDecryptData without specifying an initialization vector type now defaults to <i>IVType.Random</i> instead of <i>IVType.Hash</i>.<br />
Now uses AppSettings.UseLegacyEncryption flag to determine if the data should be signed.<br />
If decrypting and signing, now checks to ensure that the data has unhashed content (GetUnHashedData) and throws if there is no data other than the hash block. If there is no data after the signature, throws new Exception()!<br />
VerifyHashedData no longer fast-aborts when the hashed data mismatches. This makes the check take the same amount of time whether a match is found or not.</li>
<li><b>Handlers\AssemblyResourceLoader.cs</b><br />
ProcessRequest now catches any exception and morphs to an undistinguished InvalidRequest instead of leaking it out.<br />
(v4.0) Also no longer passes the assembly name to the InvalidRequest exception formatting.</li>
<li><b>Security\MachineKey.cs </b><i>(v4.0)</i><br />
Decode respects the new AppSettings.UseLegacyMachineKeyEncryption setting. [note, not AppSettings.UseLegacyEncryption]</li>
<li><b>Security\MembershipAdapter.cs</b> <i>(v4.0)</i><br />
EncryptOrDecryptData now explicitly passes false for signData to MachineKeySection.EncryptOrDecryptData to remain compatible with prior calls, whilst still showing this code has been checked.</li>
<li><b>Security\MembershipProvider.cs</b><br />
DecryptPassword and EncryptPassword now explicitly passes false for useValidationSymAlgo and signData to MachineKeySection.EncryptOrDecryptData to remain compatible with prior calls, whilst still showing this code has been checked.</li>
<li><b>UI\WebControls CheckBoxField.cs CheckBoxList.cs MailDefinition.cs ObjectDataSource.cs </b>and <b>WebControl.cs</b><br />
(v4.0) Various property attributes have been reordered [unrelated, artifact of a new compiler version?]</li>
<li><b>UI\ObjectStateFormatter.cs</b><br />
Deserialize now ignores the caught exception and now throws the <i>MacValidationError </i>without leaking out the exception details.</li>
<li><b>UI\Page.cs</b><br />
Now flags the page response as in-error for <b>any</b> exception, not just a <i>CryptographicException</i>, thus not leaking out the exception kind.</li>
<li><b>Util\AppSettings.cs</b><br />
Now exposes a <i>UseLegacyEncryption </i>setting to allow reverting to old behavior for encryption. [defaults to false]<br />
(v4.0) Now exposes a <i>UseLegacyMachineKeyEncryption </i>setting to allow reverting to old behavior for MachineKey encryption. [defaults to false]<br />
(v4.0) Now exposes a <i>ScriptResourceAllowNonJsFiles </i>setting to allow reverting to old behavior for ScriptResource encryption. [defaults to false]</li>
<li><b>Util\VersionInfo.cs</b> <i>(v4.0)</i><br />
Bumped the serialized SystemWebVersion property from 4.0.30319.1 to 4.0.30319.206</li>
</ul>
<h4>
System.Web.Extensions <i>(v4.0)</i></h4>
<ul>
<li><b>ScriptResourceHandler.cs</b><br />
ProcessRequest now catches any exception and throws an undistinguished 404 error.<br />
ProcessRequest now uses the new AppSettings.ScriptResourceAllowNonJsFiles setting to control behavior when a resource is requested that doesn't end in ".js" If enabled (<b>NOT DEFAULT</b>), then resource is allowed, otherwise now will throw an undistinguished 404 error.</li>
<li><b>VersionInfo.cs</b><br />
Bumped the serialized SystemWebVersion property from 4.0.30319.1 to 4.0.30319.206</li>
<li><b>ThisAssembly.cs</b><br />
Bumped the build revision from 1 to 206</li>
</ul>IDisposablehttp://www.blogger.com/profile/02275315449689041289noreply@blogger.com8tag:blogger.com,1999:blog-5822946.post-47100746826451604472009-09-17T02:28:00.002-05:002009-09-17T02:31:22.228-05:00And as for me? I've done nothing.My last post was 18 months ago... I deleted the two that followed it because they were time-sensitive. I've been sucked into twitter, StackOverflow, parenthood and real life.
Coming soon, I hope, will be a bunch of posts on some C# hacks and T-SQL things I've been up to... mostly ASP.Net and twitter related (e.g. huge datasets)IDisposablehttp://www.blogger.com/profile/02275315449689041289noreply@blogger.com3tag:blogger.com,1999:blog-5822946.post-34664311072647966782008-04-30T23:42:00.000-05:002008-05-01T00:07:25.095-05:00And you, sir? What have you done lately?<p>In Joel Spolsky's latest <a href="http://www.joelonsoftware.com/items/2008/05/01.html" title="bitch, moan, pity me">tirade</a>, we learn that:</p><ol><li>Microsoft is an illegal monopoly</li><li>Microsoft and Google are hiring people to pay foosball</li><li>FolderShare is a tool for uploading and download files to the internet that nobody has ever heard of.</li></ol><p>Simply put, to Joel everyone else does everything wrong. He, meanwhile, is milking a not-even-ASP bug-track application (he probably couldn't figure out how to write a ColdFusion version). Or wrapping a proprietary DynamicDNS-like wrapper around invented-elsewhere VNC client (and somehow avoiding the far more evolved UltraVNC variant). Or writing the worst little CMS monstrosity ever created. Joel is bitter about the cost of CS graduates and blames Google and Microsoft (curiously leaving Yahoo out of the mix) for driving it up.</p><p>The thing is, he's not entirely wrong... when a company can't figure out or invest in a stronger product, like <a href="http://www.attask.com/" title="be ready to commit">@Task</a> or Team Foundation, <a href="http://www.fogcreek.com/FogBugz/" title="partying on like 1999">FogBugz</a> isn't a bad compromise. When your dad needs IT support behind his random cable-modem IP, <a href="https://www.copilot.com/" title="it's not terminals, but it works">CoPilot</a> isn't a bad tool and far cheaper to use than WebEx or GoToMyPc. And well, the best I can say about CityDesk is that my technophobe wife can figure it out and Joel isn't pushing it.</p><p>Seriously, though... what have you done lately that really is worth bragging about, Joel? It's been how many years since FogBugz came out and it <strong>still</strong> can't track my billable time or hold my requirements docs? It's been years since CityDesk saw a new release, and CoPilot was written by a few interns...over a summer. What has your (self) vaunted managerial skill produced lately. You can only rest on your Excel project manager laurels for so long.</p>IDisposablehttp://www.blogger.com/profile/02275315449689041289noreply@blogger.com4tag:blogger.com,1999:blog-5822946.post-69466930281208432682008-04-15T02:30:00.001-05:002008-04-15T02:52:42.210-05:00SQL Server Data Services via cURL<p>I just noticed a really <a href="http://blogs.msdn.com/jcurrier/archive/2008/04/13/curl-ing-up-with-sql-server-data-services.aspx" title="Jeff Currier">cool article</a> on using Microsoft's new <a href="http://www.microsoft.com/sql/dataservices/default.mspx" title="Your Data, Any Place, Any Time ">SQL Server Data Services</a> which explains how to use <a href="http://curl.haxx.se/" title="command-line URL">cURL</a> at the command line to talk to the SSDS RESTful interface.</p><h3>What's cURL?</h3><p>If you've never heard of cURL, it is similar to wget in that it allows you to make HTTP requests of any web service. It can handle all the standard verbs (GET,PUT,POST,DELETE) and also supports all those lovely redirections, security and all that other nonsense. It's great for crufting up any batch/command file that could do all sorts of things as well as to ping-test REST services.</p><h3>What's SSDS</h3><p>If you've never heard of SSDS, it is similar to the Amazon SimpleDB. It offers the ability to push a database out to the public cloud and allow access from web applications, thick clients or whatever. What differentiates it from Google's APE-based access to BigTable or Amazon's S3/SimpleDB setup is that both of those systems are tuple-based (or name-value-based) non-relational databases. The SSDS stuff, on the other hand, is a Linq-based. This makes querying MUCH simpler to do.</p><p>The <strong>killer</strong> feature to me is that SSDS doesn't make you (the developer) worry about consistency. With SimpleDB or BigTable, the provider only guarantees "eventual consistency". This means that the changes you make will eventually be propogated through the Amazon/Google cloud. During the time the change was post, but not yet propogate your clients <strong>may see stale data</strong> which makes these services useable mostly for rarely-changed data.</p><p>SSDS doesn't have this restriction. Once your call is complete <strong>any access</strong> will result in the commited data being returned. This is a much simpler model to program against and it puts the replication issues squarely on the database server/service where it belongs. What remains to be seen is if Microsoft will really be able to scale this out reasonably.</p>IDisposablehttp://www.blogger.com/profile/02275315449689041289noreply@blogger.com0tag:blogger.com,1999:blog-5822946.post-85230515375428516182008-04-10T17:20:00.002-05:002008-04-10T17:21:31.302-05:00You can't hold onto nothing<p>A while back, we were looking for an easy way to count "hits" against content in a CMS-like system. For the sake of discussion, pretend we have a table called <code>ContentEntry</code> that represents the content. We decided we wanted to track the hits by-hour against a particular content entry, so that's the <code>ContentEntryPlusPlus</code> table on the right. The foreign-key is from <code>ContentEntryPlusPlus.ContentEntryID</code> to <code>ContentEntry.ID</code>.</p> <p><a href="http://lh5.ggpht.com/idisposable/R_6SptmUbMI/AAAAAAAABIU/YFa6TJbd4-8/s1600-h/ContentEntry.png"><img style="border-top-width: 0px; border-left-width: 0px; border-bottom-width: 0px; border-right-width: 0px" height="228" alt="ContentEntry" src="http://lh3.ggpht.com/idisposable/R_6SqNmUbNI/AAAAAAAABIc/RcA2TAlN5do/ContentEntry_thumb.png?imgmax=800" width="613" border="0" /></a> </p> <p>Now the trick is to insert the row if needed for a particular entry and time-slot then increment the <code>Hits</code> column. The simplest thing to do is to is check to see if the row exists, insert it if not, then do the update.  Something like this to find the row's <code>ID</code>:</p> <div class="wlWriterSmartContent" id="scid:57F11A72-B0E5-49c7-9094-E3A15BD5B5E7:505d47ce-94a2-46bf-a01a-766f5de2c6c0" style="padding-right: 0px; display: inline; padding-left: 0px; float: none; padding-bottom: 0px; margin: 0px; padding-top: 0px"><pre style="background-color:White;"><div><!--
Code highlighting produced by Actipro CodeHighlighter (freeware)
http://www.CodeHighlighter.com/
--><span style="color: #0000FF; ">SELECT</span><span style="color: #000000; "> </span><span style="color: #0000FF; ">TOP</span><span style="color: #000000; "> </span><span style="color: #800000; font-weight: bold; ">1</span><span style="color: #000000; "> ID
</span><span style="color: #0000FF; ">FROM</span><span style="color: #000000; "> dbo.ContentEntryPlusPlus
</span><span style="color: #0000FF; ">WHERE</span><span style="color: #000000; "> ContentEntryID </span><span style="color: #808080; ">=</span><span style="color: #000000; "> </span><span style="color: #008000; ">@ContentEntryID</span><span style="color: #000000; ">
</span><span style="color: #808080; ">AND</span><span style="color: #000000; "> TimeSlot </span><span style="color: #808080; ">=</span><span style="color: #000000; "> </span><span style="color: #FF00FF; ">DateAdd</span><span style="color: #000000; ">(hh, </span><span style="color: #FF00FF; ">DateDiff</span><span style="color: #000000; ">(hh, </span><span style="color: #800000; font-weight: bold; ">0</span><span style="color: #000000; ">, GetUtcDate()), </span><span style="color: #800000; font-weight: bold; ">0</span><span style="color: #000000; ">))</span></div></pre></div>
<p>Then we have to insert the row if missing:</p>
<div class="wlWriterSmartContent" id="scid:57F11A72-B0E5-49c7-9094-E3A15BD5B5E7:b86a9b25-6ecd-42fd-82a6-10a36e0c423a" style="padding-right: 0px; display: inline; padding-left: 0px; float: none; padding-bottom: 0px; margin: 0px; padding-top: 0px"><pre style="background-color:White;"><div><!--
Code highlighting produced by Actipro CodeHighlighter (freeware)
http://www.CodeHighlighter.com/
--><span style="color: #0000FF; ">INSERT</span><span style="color: #000000; "> </span><span style="color: #0000FF; ">INTO</span><span style="color: #000000; "> dbo.ContentEntryPlusPlus(ContentEntryID, TimeSlot)
</span><span style="color: #0000FF; ">VALUES</span><span style="color: #000000; "> (</span><span style="color: #008000; ">@ContentEntryID</span><span style="color: #000000; ">, </span><span style="color: #FF00FF; ">DateAdd</span><span style="color: #000000; ">(hh, </span><span style="color: #FF00FF; ">DateDiff</span><span style="color: #000000; ">(hh, </span><span style="color: #800000; font-weight: bold; ">0</span><span style="color: #000000; ">, GetUtcDate()), </span><span style="color: #800000; font-weight: bold; ">0</span><span style="color: #000000; ">))
</span><span style="color: #0000FF; ">SELECT</span><span style="color: #000000; "> </span><span style="color: #FF00FF; ">Scope_Identity</span><span style="color: #000000; ">() </span><span style="color: #0000FF; ">AS</span><span style="color: #000000; "> ID</span></div></pre></div>
<p>Then we do the update like this:</p>
<div class="wlWriterSmartContent" id="scid:57F11A72-B0E5-49c7-9094-E3A15BD5B5E7:4a0ba12d-23a9-4857-aa54-dd5f3886951e" style="padding-right: 0px; display: inline; padding-left: 0px; float: none; padding-bottom: 0px; margin: 0px; padding-top: 0px"><pre style="background-color:White;"><div><!--
Code highlighting produced by Actipro CodeHighlighter (freeware)
http://www.CodeHighlighter.com/
--><span style="color: #0000FF; ">UPDATE</span><span style="color: #000000; "> dbo.ContentEntryPlusPlus
</span><span style="color: #0000FF; ">SET</span><span style="color: #000000; "> Hits </span><span style="color: #808080; ">=</span><span style="color: #000000; "> Hits </span><span style="color: #808080; ">+</span><span style="color: #000000; "> </span><span style="color: #800000; font-weight: bold; ">1</span><span style="color: #000000; ">
</span><span style="color: #0000FF; ">WHERE</span><span style="color: #000000; "> ID </span><span style="color: #808080; ">=</span><span style="color: #000000; "> </span><span style="color: #008000; ">@ID</span><span style="color: #000000; "> </span><span style="color: #008080; ">--</span><span style="color: #008080; "> from above SELECT or INSERT's Scope_Identity()</span></div></pre></div>
<p>Obviously we have to do this inside a transaction or we could have issues and I <strong>hate </strong>multiple round-trips, so we crafted this cute statement pair to insert the row if needed and then update. Note the use of <code>INSERT FROM</code> coupled with a fake table whose row count is controlled by an <code>EXISTS</code> clause checking for the desired row. This gets executed as a single SQL command.</p>
<div class="wlWriterSmartContent" id="scid:57F11A72-B0E5-49c7-9094-E3A15BD5B5E7:e1865b54-4ea1-4742-89c6-de052c9df515" style="padding-right: 0px; display: inline; padding-left: 0px; float: none; padding-bottom: 0px; margin: 0px; padding-top: 0px"><pre style="background-color:White;"><div><!--
Code highlighting produced by Actipro CodeHighlighter (freeware)
http://www.CodeHighlighter.com/
--><span style="color: #0000FF; ">INSERT</span><span style="color: #000000; "> </span><span style="color: #0000FF; ">INTO</span><span style="color: #000000; "> dbo.ContentEntryPlusPlus(ContentEntryID, TimeSlot)
</span><span style="color: #0000FF; ">SELECT</span><span style="color: #000000; "> </span><span style="color: #0000FF; ">TOP</span><span style="color: #000000; "> </span><span style="color: #800000; font-weight: bold; ">1</span><span style="color: #000000; ">
</span><span style="color: #008000; ">@ContentEntryID</span><span style="color: #000000; "> </span><span style="color: #0000FF; ">AS</span><span style="color: #000000; "> ContentEntryID
,</span><span style="color: #FF00FF; ">DateAdd</span><span style="color: #000000; ">(hh, </span><span style="color: #FF00FF; ">DateDiff</span><span style="color: #000000; ">(hh, </span><span style="color: #800000; font-weight: bold; ">0</span><span style="color: #000000; ">, GetUtcDate()), </span><span style="color: #800000; font-weight: bold; ">0</span><span style="color: #000000; ">) </span><span style="color: #0000FF; ">AS</span><span style="color: #000000; "> TimeSlot
</span><span style="color: #0000FF; ">FROM</span><span style="color: #000000; "> (</span><span style="color: #0000FF; ">SELECT</span><span style="color: #000000; "> </span><span style="color: #800000; font-weight: bold; ">1</span><span style="color: #000000; "> </span><span style="color: #0000FF; ">AS</span><span style="color: #000000; "> FakeColumn) </span><span style="color: #0000FF; ">AS</span><span style="color: #000000; "> FakeTable
</span><span style="color: #0000FF; ">WHERE</span><span style="color: #000000; "> </span><span style="color: #808080; ">NOT</span><span style="color: #000000; "> </span><span style="color: #808080; ">EXISTS</span><span style="color: #000000; "> (</span><span style="color: #0000FF; ">SELECT</span><span style="color: #000000; "> </span><span style="color: #808080; ">*</span><span style="color: #000000; "> </span><span style="color: #0000FF; ">FROM</span><span style="color: #000000; "> dbo.ContentEntryPlusPlus
</span><span style="color: #0000FF; ">WHERE</span><span style="color: #000000; "> ContentEntryID </span><span style="color: #808080; ">=</span><span style="color: #000000; "> </span><span style="color: #008000; ">@ContentEntryID</span><span style="color: #000000; ">
</span><span style="color: #808080; ">AND</span><span style="color: #000000; "> TimeSlot </span><span style="color: #808080; ">=</span><span style="color: #000000; "> </span><span style="color: #FF00FF; ">DateAdd</span><span style="color: #000000; ">(hh, </span><span style="color: #FF00FF; ">DateDiff</span><span style="color: #000000; ">(hh, </span><span style="color: #800000; font-weight: bold; ">0</span><span style="color: #000000; ">, GetUtcDate()), </span><span style="color: #800000; font-weight: bold; ">0</span><span style="color: #000000; ">))
</span><span style="color: #0000FF; ">UPDATE</span><span style="color: #000000; "> dbo.ContentEntryPlusPlus
</span><span style="color: #0000FF; ">SET</span><span style="color: #000000; "> Hits </span><span style="color: #808080; ">=</span><span style="color: #000000; "> Hits </span><span style="color: #808080; ">+</span><span style="color: #000000; "> </span><span style="color: #800000; font-weight: bold; ">1</span><span style="color: #000000; ">
</span><span style="color: #0000FF; ">WHERE</span><span style="color: #000000; "> ContentEntryID </span><span style="color: #808080; ">=</span><span style="color: #000000; "> </span><span style="color: #008000; ">@ContentEntryID</span><span style="color: #000000; ">
</span><span style="color: #808080; ">AND</span><span style="color: #000000; "> TimeSlot </span><span style="color: #808080; ">=</span><span style="color: #000000; "> </span><span style="color: #FF00FF; ">DateAdd</span><span style="color: #000000; ">(hh, </span><span style="color: #FF00FF; ">DateDiff</span><span style="color: #000000; ">(hh, </span><span style="color: #800000; font-weight: bold; ">0</span><span style="color: #000000; ">, GetUtcDate()), </span><span style="color: #800000; font-weight: bold; ">0</span><span style="color: #000000; ">)</span></div></pre></div>
<p>This got tested and deployed, working as expected. The only problem is that every once in a while, for some particularly popular content, we would get a violation of the clustered-key's uniqueness check on the <code>ContentEntryPlusPlus</code> table. This was quite surprising, honestly as the code obviously worked when we tested it. </p>
<p>The only thing that could cause this is if the two calls executed the inner existence-check simultaneously and both decided an <code>INSERT</code> was warranted. I had assumed that locks would be acquired, and they are, for the inner <code>SELECT</code>, but since there are no rows to when this is executed, there are no rows locked, so both statements will plow on through. So, I just had to add a quick <code>WITH (HOLDLOCK)</code> hint to the inner <code>SELECT</code> and poof it works.</p>
<p>So, the moral of the story? <strong>You can't hold onto nothing...</strong></p>
<p>The final version is:</p>
<div class="wlWriterSmartContent" id="scid:57F11A72-B0E5-49c7-9094-E3A15BD5B5E7:79a4f425-4306-48b0-bc2b-0ed0f44000d2" style="padding-right: 0px; display: inline; padding-left: 0px; float: none; padding-bottom: 0px; margin: 0px; padding-top: 0px"><pre style="background-color:White;"><div><!--
Code highlighting produced by Actipro CodeHighlighter (freeware)
http://www.CodeHighlighter.com/
--><span style="color: #0000FF; ">INSERT</span><span style="color: #000000; "> </span><span style="color: #0000FF; ">INTO</span><span style="color: #000000; "> dbo.ContentEntryPlusPlus(ContentEntryID, TimeSlot)
</span><span style="color: #0000FF; ">SELECT</span><span style="color: #000000; "> </span><span style="color: #0000FF; ">TOP</span><span style="color: #000000; "> </span><span style="color: #800000; font-weight: bold; ">1</span><span style="color: #000000; ">
</span><span style="color: #008000; ">@ContentEntryID</span><span style="color: #000000; "> </span><span style="color: #0000FF; ">AS</span><span style="color: #000000; "> ContentEntryID
,</span><span style="color: #FF00FF; ">DateAdd</span><span style="color: #000000; ">(hh, </span><span style="color: #FF00FF; ">DateDiff</span><span style="color: #000000; ">(hh, </span><span style="color: #800000; font-weight: bold; ">0</span><span style="color: #000000; ">, GetUtcDate()), </span><span style="color: #800000; font-weight: bold; ">0</span><span style="color: #000000; ">) </span><span style="color: #0000FF; ">AS</span><span style="color: #000000; "> TimeSlot
</span><span style="color: #0000FF; ">FROM</span><span style="color: #000000; "> (</span><span style="color: #0000FF; ">SELECT</span><span style="color: #000000; "> </span><span style="color: #800000; font-weight: bold; ">1</span><span style="color: #000000; "> </span><span style="color: #0000FF; ">AS</span><span style="color: #000000; "> FakeColumn) </span><span style="color: #0000FF; ">AS</span><span style="color: #000000; "> FakeTable
</span><span style="color: #0000FF; ">WHERE</span><span style="color: #000000; "> </span><span style="color: #808080; ">NOT</span><span style="color: #000000; "> </span><span style="color: #808080; ">EXISTS</span><span style="color: #000000; "> (</span><span style="color: #0000FF; ">SELECT</span><span style="color: #000000; "> </span><span style="color: #808080; ">*</span><span style="color: #000000; "> </span><span style="color: #0000FF; ">FROM</span><span style="color: #000000; "> dbo.ContentEntryPlusPlus </span><span style="color: #0000FF; ">WITH</span><span style="color: #000000; "> (</span><span style="color: #0000FF; ">HOLDLOCK</span><span style="color: #000000; ">)
</span><span style="color: #0000FF; ">WHERE</span><span style="color: #000000; "> ContentEntryID </span><span style="color: #808080; ">=</span><span style="color: #000000; "> </span><span style="color: #008000; ">@ContentEntryID</span><span style="color: #000000; ">
</span><span style="color: #808080; ">AND</span><span style="color: #000000; "> TimeSlot </span><span style="color: #808080; ">=</span><span style="color: #000000; "> </span><span style="color: #FF00FF; ">DateAdd</span><span style="color: #000000; ">(hh, </span><span style="color: #FF00FF; ">DateDiff</span><span style="color: #000000; ">(hh, </span><span style="color: #800000; font-weight: bold; ">0</span><span style="color: #000000; ">, GetUtcDate()), </span><span style="color: #800000; font-weight: bold; ">0</span><span style="color: #000000; ">))
</span><span style="color: #0000FF; ">UPDATE</span><span style="color: #000000; "> dbo.ContentEntryPlusPlus
</span><span style="color: #0000FF; ">SET</span><span style="color: #000000; "> Hits </span><span style="color: #808080; ">=</span><span style="color: #000000; "> Hits </span><span style="color: #808080; ">+</span><span style="color: #000000; "> </span><span style="color: #800000; font-weight: bold; ">1</span><span style="color: #000000; ">
</span><span style="color: #0000FF; ">WHERE</span><span style="color: #000000; "> ContentEntryID </span><span style="color: #808080; ">=</span><span style="color: #000000; "> </span><span style="color: #008000; ">@ContentEntryID</span><span style="color: #000000; ">
</span><span style="color: #808080; ">AND</span><span style="color: #000000; "> TimeSlot </span><span style="color: #808080; ">=</span><span style="color: #000000; "> </span><span style="color: #FF00FF; ">DateAdd</span><span style="color: #000000; ">(hh, </span><span style="color: #FF00FF; ">DateDiff</span><span style="color: #000000; ">(hh, </span><span style="color: #800000; font-weight: bold; ">0</span><span style="color: #000000; ">, GetUtcDate()), </span><span style="color: #800000; font-weight: bold; ">0</span><span style="color: #000000; ">)</span></div></pre></div>IDisposablehttp://www.blogger.com/profile/02275315449689041289noreply@blogger.com2tag:blogger.com,1999:blog-5822946.post-71999195415035248562008-03-09T03:23:00.004-05:002008-03-09T04:16:42.269-05:00Sometimes you make a hash of things.<p>So, hash has been on my mind lately. No, not <a title="hashish" href="http://en.wikipedia.org/wiki/Hashish">that kind</a> of hash, or <a title="mmmm, now that I'm thinking of it, sounds tasty" href="http://en.wikipedia.org/wiki/Hash_%28food%29">that kind</a> either. First, there was last week, when I installed Internet Explorer 8 beta 1. I was reading the release notes and was amazed to find that # (you know, octothorpe, pound sign) was not considered part of the URL by this version. Thus you can't link directly to a <code>name</code>d element on a page. Eeew!</p> <p>Then today, <a title="I Web, Therefore I Am" href="http://www.iwebthereforeiam.com/">Hugh Brown</a> dropped a comment on my <a href="http://musingmarc.blogspot.com/2007/08/vtos-rtos-and-gethashcode-oh-my.html">diatribe post about value-types, reference-types, Equals and GetHashCode</a>. The post has been live for many months now, and has quite a bit of Google juice. Until now, nobody has ever quibbled with the stuff I wrote, but Hugh had some interesting observations.</p> <h3>First the little stuff</h3> <p>In a minor part of his comment, he was surprised by the many overloads of GetHashCode that I suggest, wondering why I didn't just always expect callers to use the params int[] version. Quite simply, this is because by providing several overloads for a small number of arguments (5 in my example), I avoid paying the cost of allocating the array of integers and copying the values for each call to the <code>CombineHashCodes</code>. While this may seem like a trivial savings, remember that <code>GetHashCode</code> is called many times when dealing with <code>HashTable</code> collections and thus it is worth it to provide expedited code paths for the more common usages. Additional savings inside the <code>CombineHashCodes</code> method are garnered by avoiding the loop setup/iteration overhead. Finally, in optimized builds, these simpler method calls will be inlined by the compiler and/or JIT, where methods having loops in the body are never inlined (in CLR releases thus far). It is worth noting that the .Net runtime implementation does the same thing for <code>System.Web.Util.HashCodeCombiner</code> and <code>System.String.Format</code>.</p> <h3>To the meat of the comment</h3> <p>The main body of his comment was that my code actually didn't return useful values. That concerned me a lot. Given his use of Python and inlined implementation, I had to write my own test jig. Unfortunately it confirmed his complaint. On the one hand, the values he was using to test were not normal values you would expect from <code>GetHashCode</code>. Normally <code>GetHashCode</code> values are well-distributed across the entire 32-bit range of an <code>Int32</code>. He was using sequential, smallish, numbers which was skewing the result oddly. That said, the values SHOULD have been different for very-similar inputs. I delved a little into the code I originally wrote and found that what's on the web page does NOT match what is <strong>now</strong> in use in the BCL's internal code to combine hash codes (which is where I got the idea of left-shifting by 5 bits before XORing). I think that my code was originally based on the 1.1 BCL but I'm not really sure.</p> <p>In the .Net 2.0 version, there's a class called <code>System.Web.Util.HashCodeCombiner</code> that actually reflects essentially the same technique as my code, with one huge and very significant difference. Where I simply left-shift the running hash code by 5 bits and then XOR in the next value, they are doing the left-shift <strong>and also adding in the running hash</strong>, then doing the XOR.</p> <h3>Why so shifty, anyway?</h3> <p>You might be wondering why do the left shift in the first place. The simple answer is that by doing a left-shift by some number of bits, we preserve the low order bits of the running hash somewhat. This prevents the incoming value from XORing away all the significance of the bits thus far, and also insures that low-byte-only intermediate hash codes don't simply cancel each other out. By shifting left 5 digits, we're simply multiplying by 32 (and thus preserving the lowest 5 digits). Then the original running hash value is added in on more time, making the effective multiplier 33. This isn't far off from Hugh's suggestion of multiplying by 37, while being significantly faster in the binary world of computers. Once the shift and add (e.g. multiplication by 33) is completed, the XOR of the new values results in much better distribution of the final value.</p> <p>I've updated my code in the <a title="My downloads" href="http://idisposable.googlepages.com/downloads">Utilities library</a>, and I'm going back to the original post to point to this post and the new code. So, I owe you one, Hugh...and <strong>maybe Microsoft does too</strong> because while I was reviewing their code in the newly released BCL source code, I found a very unexpected implementation. This is the snippet in question:<pre> internal static int CombineHashCodes(int h1, int h2) {
return ((h1 << 5) + h1) ^ h2;
}
internal static int CombineHashCodes(int h1, int h2, int h3) {
return CombineHashCodes(CombineHashCodes(h1, h2), h3);
}
internal static int CombineHashCodes(int h1, int h2, int h3, int h4) {
return CombineHashCodes(CombineHashCodes(h1, h2), CombineHashCodes(h3, h4));
}
internal static int CombineHashCodes(int h1, int h2, int h3, int h4, int h5) {
return CombineHashCodes(CombineHashCodes(h1, h2, h3, h4), h5);
}</pre>Did you see the oddity? That implementation taking 4 values does its work by calling the two-value one three times. Once to combine the first pair (h1 and h2) of arguments, once to combine the second pair (h3 and h4), then finally to combine the two intermediate values. That's a bit different than doing what the 3-value and 5-value overloads use. I personally think it should have called the 2-value against output of the 3-value to combine the 4th value (h4). That would be more like what the 3-value and 5-value overload do. In other words, the method should be:<pre> internal static int CombineHashCodes(int h1, int h2, int h3, int h4) {
return CombineHashCodes(CombineHashCodes(h1, h2, h3), h4);
}</pre>
<p>Perhaps they don't care that the values are inconsistent, especially since they don't provide a combiner that takes a <code>params int[]</code> overload, but imagine if I had blindly copied that code and you got two different values from this:<pre> Console.WriteLine("Testing gotcha:");<br> Console.WriteLine(String.Format("1,2: {0:x}", Utilities.CombineHashCodes(1, 2)));<br> Console.WriteLine(String.Format("1,2,3: {0:x}", Utilities.CombineHashCodes(1, 2, 3)));<br> Console.WriteLine(String.Format("1,2,3,4: {0:x}", Utilities.CombineHashCodes(1, 2, 3, 4)));<br> Console.WriteLine(String.Format("1,2,3,4,5: {0:x}", Utilities.CombineHashCodes(1, 2, 3, 4, 5)));
<br> Console.WriteLine(String.Format("[1,2]: {0:x}", Utilities.CombineHashCodes(new int[] { 1, 2 })));<br> Console.WriteLine(String.Format("[1,2,3]: {0:x}", Utilities.CombineHashCodes(new int[] { 1, 2, 3 })));<br> Console.WriteLine(String.Format("[1,2,3,4]: {0:x}", Utilities.CombineHashCodes(new int[] { 1, 2, 3, 4 })));<br> Console.WriteLine(String.Format("[1,2,3,4,5]: {0:x}", Utilities.CombineHashCodes(new int[] { 1, 2, 3, 4, 5 })));</pre>
<h3>Where we are at now</h3>
<p>Here is the revised version of the <code>CombineHashCodes</code> methods from my <a title="My downloads" href="http://idisposable.googlepages.com/downloads">Utilities library</a><pre class="code" style="background: #000000"><span style="color: #e0e0e0"> </span><span style="color: #8080c0">public static partial class </span><span style="color: #c7c7f1">Utilities
</span><span style="color: #e0e0e0">{
</span><span style="color: gray"> </span><span style="color: #8080c0">public static int </span><span style="color: #fef1ac">CombineHashCodes</span><span style="color: #e0e0e0">(</span><span style="color: #8080c0">params int</span><span style="color: #e0e0e0">[] </span><span style="color: #fef1ac">hashes</span><span style="color: #e0e0e0">)
{
</span><span style="color: #8080c0">int </span><span style="color: #fef1ac">hash </span><span style="color: #e0e0e0">= 0;
</span><span style="color: #8080c0">for </span><span style="color: #e0e0e0">(</span><span style="color: #8080c0">int </span><span style="color: #fef1ac">index </span><span style="color: #e0e0e0">= 0; </span><span style="color: #fef1ac">index </span><span style="color: #e0e0e0">< </span><span style="color: #fef1ac">hashes</span><span style="color: #e0e0e0">.</span><span style="color: #fef1ac">Length</span><span style="color: #e0e0e0">; </span><span style="color: #fef1ac">index</span><span style="color: #e0e0e0">++)
{
</span><span style="color: #fef1ac">hash </span><span style="color: #e0e0e0">= (</span><span style="color: #fef1ac">hash </span><span style="color: #e0e0e0"><< 5) + </span><span style="color: #fef1ac">hash</span><span style="color: #e0e0e0">;
</span><span style="color: #fef1ac">hash </span><span style="color: #e0e0e0">^= </span><span style="color: #fef1ac">hashes</span><span style="color: #e0e0e0">[</span><span style="color: #fef1ac">index</span><span style="color: #e0e0e0">];
}
</span><span style="color: #8080c0">return </span><span style="color: #fef1ac">hash</span><span style="color: #e0e0e0">;
}
</span><span style="color: gray"> </span><span style="color: #8080c0">private static int </span><span style="color: #fef1ac">GetEntryHash</span><span style="color: #e0e0e0">(</span><span style="color: #8080c0">object </span><span style="color: #fef1ac">entry</span><span style="color: #e0e0e0">)
{
</span><span style="color: #8080c0">int </span><span style="color: #fef1ac">entryHash </span><span style="color: #e0e0e0">= 0x61E04917; </span><span style="color: #c080c0">// slurped from .Net runtime internals...
</span><span style="color: #8080c0">if </span><span style="color: #e0e0e0">(</span><span style="color: #fef1ac">entry </span><span style="color: #e0e0e0">!= </span><span style="color: #8080c0">null</span><span style="color: #e0e0e0">)
{
</span><span style="color: #8080c0">object</span><span style="color: #e0e0e0">[] </span><span style="color: #fef1ac">subObjects </span><span style="color: #e0e0e0">= </span><span style="color: #fef1ac">entry </span><span style="color: #8080c0">as object</span><span style="color: #e0e0e0">[];
</span><span style="color: #8080c0">if </span><span style="color: #e0e0e0">(</span><span style="color: #fef1ac">subObjects </span><span style="color: #e0e0e0">!= </span><span style="color: #8080c0">null</span><span style="color: #e0e0e0">)
{
</span><span style="color: #fef1ac">entryHash </span><span style="color: #e0e0e0">= </span><span style="color: #c7c7f1">Utilities</span><span style="color: #e0e0e0">.</span><span style="color: #fef1ac">CombineHashCodes</span><span style="color: #e0e0e0">(</span><span style="color: #fef1ac">subObjects</span><span style="color: #e0e0e0">);
}
</span><span style="color: #8080c0">else
</span><span style="color: #e0e0e0">{
</span><span style="color: #fef1ac">entryHash </span><span style="color: #e0e0e0">= </span><span style="color: #fef1ac">entry</span><span style="color: #e0e0e0">.</span><span style="color: #fef1ac">GetHashCode</span><span style="color: #e0e0e0">();
}
}
</span><span style="color: #8080c0">return </span><span style="color: #fef1ac">entryHash</span><span style="color: #e0e0e0">;
}
</span><span style="color: gray"> </span><span style="color: #8080c0">public static int </span><span style="color: #fef1ac">CombineHashCodes</span><span style="color: #e0e0e0">(</span><span style="color: #8080c0">params object</span><span style="color: #e0e0e0">[] </span><span style="color: #fef1ac">objects</span><span style="color: #e0e0e0">)
{
</span><span style="color: #8080c0">int </span><span style="color: #fef1ac">hash </span><span style="color: #e0e0e0">= 0;
</span><span style="color: #8080c0">for </span><span style="color: #e0e0e0">(</span><span style="color: #8080c0">int </span><span style="color: #fef1ac">index </span><span style="color: #e0e0e0">= 0; </span><span style="color: #fef1ac">index </span><span style="color: #e0e0e0">< </span><span style="color: #fef1ac">objects</span><span style="color: #e0e0e0">.</span><span style="color: #fef1ac">Length</span><span style="color: #e0e0e0">; </span><span style="color: #fef1ac">index</span><span style="color: #e0e0e0">++)
{
</span><span style="color: #fef1ac">hash </span><span style="color: #e0e0e0">= (</span><span style="color: #fef1ac">hash </span><span style="color: #e0e0e0"><< 5) + </span><span style="color: #fef1ac">hash</span><span style="color: #e0e0e0">;
</span><span style="color: #fef1ac">hash </span><span style="color: #e0e0e0">^= </span><span style="color: #fef1ac">GetEntryHash</span><span style="color: #e0e0e0">(</span><span style="color: #fef1ac">objects</span><span style="color: #e0e0e0">[</span><span style="color: #fef1ac">index</span><span style="color: #e0e0e0">]);
}
</span><span style="color: #8080c0">return </span><span style="color: #fef1ac">hash</span><span style="color: #e0e0e0">;
}
</span><span style="color: gray"> </span><span style="color: #8080c0">public static int </span><span style="color: #fef1ac">CombineHashCodes</span><span style="color: #e0e0e0">(</span><span style="color: #8080c0">int </span><span style="color: #fef1ac">hash1</span><span style="color: #e0e0e0">, </span><span style="color: #8080c0">int </span><span style="color: #fef1ac">hash2</span><span style="color: #e0e0e0">)
{
</span><span style="color: #8080c0">return </span><span style="color: #e0e0e0">((</span><span style="color: #fef1ac">hash1 </span><span style="color: #e0e0e0"><< 5) + </span><span style="color: #fef1ac">hash1</span><span style="color: #e0e0e0">)
^ </span><span style="color: #fef1ac">hash2</span><span style="color: #e0e0e0">;
}
</span><span style="color: gray"> </span><span style="color: #8080c0">public static int </span><span style="color: #fef1ac">CombineHashCodes</span><span style="color: #e0e0e0">(</span><span style="color: #8080c0">int </span><span style="color: #fef1ac">hash1</span><span style="color: #e0e0e0">, </span><span style="color: #8080c0">int </span><span style="color: #fef1ac">hash2</span><span style="color: #e0e0e0">, </span><span style="color: #8080c0">int </span><span style="color: #fef1ac">hash3</span><span style="color: #e0e0e0">)
{
</span><span style="color: #8080c0">int </span><span style="color: #fef1ac">hash </span><span style="color: #e0e0e0">= </span><span style="color: #fef1ac">CombineHashCodes</span><span style="color: #e0e0e0">(</span><span style="color: #fef1ac">hash1</span><span style="color: #e0e0e0">, </span><span style="color: #fef1ac">hash2</span><span style="color: #e0e0e0">);
</span><span style="color: #8080c0">return </span><span style="color: #e0e0e0">((</span><span style="color: #fef1ac">hash </span><span style="color: #e0e0e0"><< 5) + </span><span style="color: #fef1ac">hash</span><span style="color: #e0e0e0">)
^ </span><span style="color: #fef1ac">hash3</span><span style="color: #e0e0e0">;
}
</span><span style="color: gray"> </span><span style="color: #8080c0">public static int </span><span style="color: #fef1ac">CombineHashCodes</span><span style="color: #e0e0e0">(</span><span style="color: #8080c0">int </span><span style="color: #fef1ac">hash1</span><span style="color: #e0e0e0">, </span><span style="color: #8080c0">int </span><span style="color: #fef1ac">hash2</span><span style="color: #e0e0e0">, </span><span style="color: #8080c0">int </span><span style="color: #fef1ac">hash3</span><span style="color: #e0e0e0">, </span><span style="color: #8080c0">int </span><span style="color: #fef1ac">hash4</span><span style="color: #e0e0e0">)
{
</span><span style="color: #8080c0">int </span><span style="color: #fef1ac">hash </span><span style="color: #e0e0e0">= </span><span style="color: #fef1ac">CombineHashCodes</span><span style="color: #e0e0e0">(</span><span style="color: #fef1ac">hash1</span><span style="color: #e0e0e0">, </span><span style="color: #fef1ac">hash2</span><span style="color: #e0e0e0">, </span><span style="color: #fef1ac">hash3</span><span style="color: #e0e0e0">);
</span><span style="color: #8080c0">return </span><span style="color: #e0e0e0">((</span><span style="color: #fef1ac">hash </span><span style="color: #e0e0e0"><< 5) + </span><span style="color: #fef1ac">hash</span><span style="color: #e0e0e0">)
^ </span><span style="color: #fef1ac">hash4</span><span style="color: #e0e0e0">;
}
</span><span style="color: gray"> </span><span style="color: #8080c0">public static int </span><span style="color: #fef1ac">CombineHashCodes</span><span style="color: #e0e0e0">(</span><span style="color: #8080c0">int </span><span style="color: #fef1ac">hash1</span><span style="color: #e0e0e0">, </span><span style="color: #8080c0">int </span><span style="color: #fef1ac">hash2</span><span style="color: #e0e0e0">, </span><span style="color: #8080c0">int </span><span style="color: #fef1ac">hash3</span><span style="color: #e0e0e0">, </span><span style="color: #8080c0">int </span><span style="color: #fef1ac">hash4</span><span style="color: #e0e0e0">, </span><span style="color: #8080c0">int </span><span style="color: #fef1ac">hash5</span><span style="color: #e0e0e0">)
{
</span><span style="color: #8080c0">int </span><span style="color: #fef1ac">hash </span><span style="color: #e0e0e0">= </span><span style="color: #fef1ac">CombineHashCodes</span><span style="color: #e0e0e0">(</span><span style="color: #fef1ac">hash1</span><span style="color: #e0e0e0">, </span><span style="color: #fef1ac">hash2</span><span style="color: #e0e0e0">, </span><span style="color: #fef1ac">hash3</span><span style="color: #e0e0e0">, </span><span style="color: #fef1ac">hash4</span><span style="color: #e0e0e0">);
</span><span style="color: #8080c0">return </span><span style="color: #e0e0e0">((</span><span style="color: #fef1ac">hash </span><span style="color: #e0e0e0"><< 5) + </span><span style="color: #fef1ac">hash</span><span style="color: #e0e0e0">)
^ </span><span style="color: #fef1ac">hash5</span><span style="color: #e0e0e0">;
}
</span><span style="color: gray"> </span><span style="color: #8080c0">public static int </span><span style="color: #fef1ac">CombineHashCodes</span><span style="color: #e0e0e0">(</span><span style="color: #8080c0">object </span><span style="color: #fef1ac">obj1</span><span style="color: #e0e0e0">, </span><span style="color: #8080c0">object </span><span style="color: #fef1ac">obj2</span><span style="color: #e0e0e0">)
{
</span><span style="color: #8080c0">return </span><span style="color: #fef1ac">CombineHashCodes</span><span style="color: #e0e0e0">(</span><span style="color: #fef1ac">obj1</span><span style="color: #e0e0e0">.</span><span style="color: #fef1ac">GetHashCode</span><span style="color: #e0e0e0">()
, </span><span style="color: #fef1ac">obj2</span><span style="color: #e0e0e0">.</span><span style="color: #fef1ac">GetHashCode</span><span style="color: #e0e0e0">());
}
</span><span style="color: gray"> </span><span style="color: #8080c0">public static int </span><span style="color: #fef1ac">CombineHashCodes</span><span style="color: #e0e0e0">(</span><span style="color: #8080c0">object </span><span style="color: #fef1ac">obj1</span><span style="color: #e0e0e0">, </span><span style="color: #8080c0">object </span><span style="color: #fef1ac">obj2</span><span style="color: #e0e0e0">, </span><span style="color: #8080c0">object </span><span style="color: #fef1ac">obj3</span><span style="color: #e0e0e0">)
{
</span><span style="color: #8080c0">return </span><span style="color: #fef1ac">CombineHashCodes</span><span style="color: #e0e0e0">(</span><span style="color: #fef1ac">obj1</span><span style="color: #e0e0e0">.</span><span style="color: #fef1ac">GetHashCode</span><span style="color: #e0e0e0">()
, </span><span style="color: #fef1ac">obj2</span><span style="color: #e0e0e0">.</span><span style="color: #fef1ac">GetHashCode</span><span style="color: #e0e0e0">()
, </span><span style="color: #fef1ac">obj3</span><span style="color: #e0e0e0">.</span><span style="color: #fef1ac">GetHashCode</span><span style="color: #e0e0e0">());
}
</span><span style="color: gray"> </span><span style="color: #8080c0">public static int </span><span style="color: #fef1ac">CombineHashCodes</span><span style="color: #e0e0e0">(</span><span style="color: #8080c0">object </span><span style="color: #fef1ac">obj1</span><span style="color: #e0e0e0">, </span><span style="color: #8080c0">object </span><span style="color: #fef1ac">obj2</span><span style="color: #e0e0e0">, </span><span style="color: #8080c0">object </span><span style="color: #fef1ac">obj3</span><span style="color: #e0e0e0">, </span><span style="color: #8080c0">object </span><span style="color: #fef1ac">obj4</span><span style="color: #e0e0e0">)
{
</span><span style="color: #8080c0">return </span><span style="color: #fef1ac">CombineHashCodes</span><span style="color: #e0e0e0">(</span><span style="color: #fef1ac">obj1</span><span style="color: #e0e0e0">.</span><span style="color: #fef1ac">GetHashCode</span><span style="color: #e0e0e0">()
, </span><span style="color: #fef1ac">obj2</span><span style="color: #e0e0e0">.</span><span style="color: #fef1ac">GetHashCode</span><span style="color: #e0e0e0">()
, </span><span style="color: #fef1ac">obj3</span><span style="color: #e0e0e0">.</span><span style="color: #fef1ac">GetHashCode</span><span style="color: #e0e0e0">()
, </span><span style="color: #fef1ac">obj4</span><span style="color: #e0e0e0">.</span><span style="color: #fef1ac">GetHashCode</span><span style="color: #e0e0e0">());
}
</span><span style="color: gray"> </span><span style="color: #8080c0">public static int </span><span style="color: #fef1ac">CombineHashCodes</span><span style="color: #e0e0e0">(</span><span style="color: #8080c0">object </span><span style="color: #fef1ac">obj1</span><span style="color: #e0e0e0">, </span><span style="color: #8080c0">object </span><span style="color: #fef1ac">obj2</span><span style="color: #e0e0e0">, </span><span style="color: #8080c0">object </span><span style="color: #fef1ac">obj3</span><span style="color: #e0e0e0">, </span><span style="color: #8080c0">object </span><span style="color: #fef1ac">obj4</span><span style="color: #e0e0e0">, </span><span style="color: #8080c0">object </span><span style="color: #fef1ac">obj5</span><span style="color: #e0e0e0">)
{
</span><span style="color: #8080c0">return </span><span style="color: #fef1ac">CombineHashCodes</span><span style="color: #e0e0e0">(</span><span style="color: #fef1ac">obj1</span><span style="color: #e0e0e0">.</span><span style="color: #fef1ac">GetHashCode</span><span style="color: #e0e0e0">()
, </span><span style="color: #fef1ac">obj2</span><span style="color: #e0e0e0">.</span><span style="color: #fef1ac">GetHashCode</span><span style="color: #e0e0e0">()
, </span><span style="color: #fef1ac">obj3</span><span style="color: #e0e0e0">.</span><span style="color: #fef1ac">GetHashCode</span><span style="color: #e0e0e0">()
, </span><span style="color: #fef1ac">obj4</span><span style="color: #e0e0e0">.</span><span style="color: #fef1ac">GetHashCode</span><span style="color: #e0e0e0">()
, </span><span style="color: #fef1ac">obj5</span><span style="color: #e0e0e0">.</span><span style="color: #fef1ac">GetHashCode</span><span style="color: #e0e0e0">());
}
}</span>
</pre>IDisposablehttp://www.blogger.com/profile/02275315449689041289noreply@blogger.com7tag:blogger.com,1999:blog-5822946.post-74006411954508362272008-01-02T01:01:00.001-06:002008-01-02T01:01:12.702-06:00Random thoughts to start the year.<blockquote> <p><em>Little bunny FoFo, hopping through the forest, s</em><em>cooping up the field mice and bopping them on the heads.</em></p> <p><em>Down came the good fairy, "Little bunny FoFo, I don't want to see you scooping up the field mice and bopping them on the heads. I'll give you three chances and then I'll turn you into a goon".</em></p></blockquote> <ol> <li>The Blues deserve a good bounce. I just finished watching the Blues v. Stars game and there's no question that they outplayed the Stars all but the first couple minutes. That crappy bounce off Tkachuk's skate beat us. That's all. <li>Xen's bout with RSV teaches a couple lessons. First, no matter how hard the nights are, you will miss your boy's loud cry when it is replaced by a weak wail of despair. My soul can't handle that kind of trial very often and I thank God that he knows what I can bear. It's simple, when you can't do anything to help, you feel useless... when everything you do (sucking his nose clear, pounding the phlegm out of his lungs) actually make your child cry more, it's HARD. Second, there are things that make you remember where you are in life... I'm a senior developer, the old-wise-guy at church, and a BABY as a dad. I don't know squat. <li>When you need something that isn't .Net all the way, expect it to be hard. In this case, Subversion to Team Foundation tools SUCK. And the Subversion paradigm expects you to Alt.Net it and NOT use TFS, even though it is several dozen times better a tool. Expect me to be announcing something based on the TFS Migration Toolkit soon, cause damn sure Microsoft isn't going to bother. <li>Be VERY careful what skills you teach your children. No matter if they are good, or bad, they will be used against you.<br>The other day, at a Blues home game; my 4 year old daughter, Arianna, was squirming a bit in the seat. I told her that if she didn't stop I was going to turn her into a goon. The next whistle (she's a GOOD hockey fan) she asks, "Dad, are fairies real?". <br>My spidey sense being dull, I answered, "No, they're just like Santa Claus, just a character."<br>She replies with <strong>no</strong> delay, "Then you can't turn me into a goon."</li></ol> IDisposablehttp://www.blogger.com/profile/02275315449689041289noreply@blogger.com0tag:blogger.com,1999:blog-5822946.post-14465749299249757032007-11-18T23:35:00.000-06:002007-11-18T23:51:11.356-06:00Individual label RSS subscriptions now available<p>It seems there are several not-very-overlapping audiences for this blog. There are people reading for the <a href="http://musingmarc.blogspot.com/search/label/SQL">SQL</a> stuff, especially the <a href="http://musingmarc.blogspot.com/search/label/DateTime">datetime</a> related stuff. There are people reading for the <a href="http://musingmarc.blogspot.com/search/label/lightweight%20code%20generation">Lightweight Code Generation</a> stuff, especially the <a href="http://musingmarc.blogspot.com/search/label/DynamicMethod">DynamicMethod/DynamicSorter</a> library. Then there are the people hunting down information about the <a href="http://musingmarc.blogspot.com/search/label/RssToolkit">RSSToolkit</a> library. Finally, there's the people following the recent <a href="http://musingmarc.blogspot.com/search/label/UriTemplate">URITemplate</a> library.</p><p>Since many of you visitors seem to have specific interestes, I've added the ability to subscribe to individual labels applied to the posts via the excellent tip given by Daniel Cazzulino in his <a href="http://feeds.feedburner.com/~r/DanielCazzulino/~3/185857822/41836.aspx" title="Blogger: how to provide label/tag feeds">instructional posting</a>.</p><p>Just check out the labels listing on the right-side navigation. Oh, if you only read via a feed, this <em>might</em> be worth a read of the actual page.</p>IDisposablehttp://www.blogger.com/profile/02275315449689041289noreply@blogger.com0tag:blogger.com,1999:blog-5822946.post-27171970427872499212007-11-13T15:54:00.001-06:002007-11-18T23:52:15.040-06:00Post as if you care...<p>I'm in... </p> <ul> <li> <h4>Say <em>Everything</em> As If Speaking To <em>Everyone </em> </h4> <p>(because you are)</p> <li> <h4>If You <em>Must</em> Be A <em>Jerk</em>, Don't Be An <em>Anonymous</em> One</h4>(because that's cowardly) <li> <h4><em>Encourage</em> Others To <em>Abide</em> By This Code</h4>(because it's neighborly, plus recursive rules are fun) <li> <h4>When <em>Others</em> Don't Care To Abide, <b><em>Ignore Them</em></b></h4>(because they're not worthy of your time) </li></ul> <p><a href="http://www.asimplecode.com/">A Simple Code - Web Karma, Distilled</a></p>IDisposablehttp://www.blogger.com/profile/02275315449689041289noreply@blogger.com0tag:blogger.com,1999:blog-5822946.post-7839126068581522822007-10-30T16:23:00.001-05:002007-11-07T17:41:54.766-06:00Eye-Fi is on sale<p>I can't tell you how happy the last few days have made my inner geek. Last week the Chumby started shipping and today the be-far-coolest idea ever is available for order.</p><p>Do you have a digital camera? Snap a lot of shots? Forget to get around to uploading them to your PC and your online site of choice Have we got a solution for you, just get a Eye-Fi SD memory card, configure from your PC/Mac and then install it in your camera. It'll store 2GB of pictures and every time it gets near to a wi-fi network that you have configured it to use, poof instant uploads to your online site. This baby supports all the players (except WinkFlash, what's up with THAT?).</p><p>For those of you with CF cards instead... PFFFTT!</p><p><a href="http://www.eye.fi/">Eye-Fi</a></p>IDisposablehttp://www.blogger.com/profile/02275315449689041289noreply@blogger.com1tag:blogger.com,1999:blog-5822946.post-51239790151926957072007-10-26T19:51:00.001-05:002011-07-21T16:42:47.835-05:00New release of UriTemplate / UriPattern library<p>Today I released a new version of the UriPattern and UriTemplate library on CodePlex (<a href="http://musingmarc.blogspot.com/2007/06/uritemplate-project-on-codeplex.html">previously announced here</a>). There are two changes in this release:</p><ol><li>A bug reported by Darrel Miller where the meta character that have special meaning in Regex expressions are not properly escaped. I inherited this bug in the original implementation I based the library on, but no excuses, this was stupid. Sorry to anyone bit by this.</li><li>I've added the ability to specify that a UriPattern should be compiled. This should speed up patterns that are used very frequently.</li></ol><p>Pick up <a href="http://www.codeplex.com/UriTemplate">Release 1.1 on CodePlex</a></p>IDisposablehttp://www.blogger.com/profile/02275315449689041289noreply@blogger.com0tag:blogger.com,1999:blog-5822946.post-11426083529005628512007-10-09T17:22:00.000-05:002007-11-18T23:52:41.434-06:00Can a Lack of Sleep Set Back Your Child's Cognitive Abilities?<p>With a new baby around, you can imagine that our family's sleep patterns are changing. To say that we are tired misses the point entirely... we're all a "bit slow" round the house. Arianna doesn't want to get up for the Montessori school that she dearly loves to go to, Beth is stressed and struggling with emotion... and mellow me is actually not catching those "snaps of testosterone". That's just the emotionaly impact... the cognitive impact is <strong>much worse</strong>. I've found it difficult to grok code-review changes that occured in the last 5 days at work... I couldn't even recognize a bad web.config connection-string issue (something that would have jumped out before the problem description was finished a mere week ago). It's getting better, though... today is better than yesterday by far... and the biggest difference is in how much sleep we've gotten. I can easily see the pattern in myself--I even might generalize to Beth--but did I extend this to a general behavior pattern for Arianna, or kids in general? I am not that smart (today?).</p><p>Today, I read an article by Po Bronson, who authored an article a while back that really resounded with me. <a href="http://musingmarc.blogspot.com/2007/03/how-not-to-talk-to-your-kids.html">I wrote about it here</a> back in March. This new article shows astonishing evidence for the direct link between how much sleep a child gets and thier cognitive ability the next (and following days). In one study of 77 kids (half asked to stay up a little later and half asked to go to bed a little earlier) the resulting merely one hour difference in the amount of sleep showed the same cognitive difference after three days as that between an average 4th and 6th grader. In other words, three hours of sleep difference cost <strong>two years</strong> worth of cognitive ability.</p><p>So let, no <strong>MAKE</strong>, your kids (and you) get that extra sleep. Read more at:
<a href="http://nymag.com/news/features/38951/">Can a Lack of Sleep Set Back Your Child's Cognitive Abilities?</a></p>IDisposablehttp://www.blogger.com/profile/02275315449689041289noreply@blogger.com1tag:blogger.com,1999:blog-5822946.post-15166894608247538822007-10-04T22:41:00.002-05:002008-04-10T18:48:36.599-05:00Oh no, there's another Brooks in the world!<p>I am happy to announce the birth of Xavier Eli Brooks at 1322 of October 4th.</p> <p>After faking us out by turning himself around the night before the inversion, he resumed his (dad mirroring) ways and refused to turn the crown fully upside down. After 12 hours of Cervidil and 18 hours of contractions standing on his ear, he wasn't coming any closer to finding the stage door so we opened a new one just to his right.</p><p>He emerged warping space-time at a mass of 7 pounds 6 ounces, and a length of 19 3/4 inches, not that those numbers actually tell you anything about him.</p><p>Beth and baby are both fine, thanks for asking.</p> <p><a title="Brooks... Xavier Eli Brooks" href="http://picasaweb.google.com/idisposable/XavierEliBrooksIsBorn/photo#5117678976782819538" atomicselection="true">Brooks... Xen Brooks</a></p>IDisposablehttp://www.blogger.com/profile/02275315449689041289noreply@blogger.com1tag:blogger.com,1999:blog-5822946.post-91968958121968193862007-09-07T16:02:00.001-05:002007-09-07T16:04:32.452-05:00A sad wrinkle...<p> I will always remember the feeling of wonder that overtook me as I read "A Wrinkle in Time" for the first time in 1971... a book born of a fertile mind the same year I was born has shaped me ever since. We've lost a wonderful person today.</p><p><a href="http://www.nytimes.com/2007/09/08/books/07cnd-lengle.html?_r=1&hp&oref=slogin">Madeleine L’Engle, Children’s Writer, Is Dead - New York Times</a></p>IDisposablehttp://www.blogger.com/profile/02275315449689041289noreply@blogger.com0tag:blogger.com,1999:blog-5822946.post-88958811396399690462007-08-17T17:21:00.002-05:002008-03-09T03:30:16.470-05:00VTOs, RTOs and GetHashCode() -- oh, my!<p><strong>UPDATED</strong>:On 9 March, 2008, I fixed some issues with this posting due to comments from Hugh Brown, make <strong>sure</strong> you use the read the follow-up post <a href="http://musingmarc.blogspot.com/2008/03/sometimes-you-make-hash-of-things.html" title="Oops...">Sometimes you make a hash of things</a>.</p>
<h3>Introduction</h3> <p>Checking out a new blog today [<a title="Good stuff from another .Net hacker" href="http://ralinx.wordpress.com/" target="_blank">Davy Brion's Blog</a>] I stumbled across a very nice entry about <a title="Immutable objects mean that the values can't be changed without yielding a new object" href="http://ralinx.wordpress.com/2007/07/08/implementing-a-value-object/" target="_blank">Implementing A Value Object</a>. Go read that now if you don't know what a value object is, what immutable means or why it's good.</p> <h3>Identity is who you are</h3> <p>What I want to talk about is GetHashCode() as used with value-type objects (e.g. <code>struct</code> in C#) but to do that, I really need to talk about the difference between reference-type objects (<abbr title="reference-type object">RTO</abbr>s from here out) vs. value-type objects (<abbr title="value-type object">VTO</abbr>s from here out). Feel free to skip down if this is old hat to you.</p> <p>What's important to realize is that if your are a reference-type object, your identity revolved around "where you are". This is expressed, in terms of .Net, by the fact that you have the same reference handle/memory address. The problem with this is that you might have an <code>Person</code> object that <strong>currently </strong>represents me and thus has the <code>FirstName</code> property == "Marc" and <code>LastName</code> property == "Brooks". If I give you a reference to that <code>Person</code> object and you change the <code>FirstName</code> property to "Charles", you're suddenly talking about my father. What's dangerous about this is that you have changed the underlying object to which I gave you a reference, thus my reference <strong>also</strong> now seems to be my father.</p> <p>On the other hand, if I gave you a copy of the original <code>Person</code> object (perhaps via a <code>Clone()</code> operation), then you can change any property you wish and I will never know. This is good, if that's what you intend. Your personal copy of the object is <strong>not</strong> my copy of the object, they have different <strong>physical </strong>identities, even though they might initially share the same <strong>logical</strong> identity. To me, it's much like the difference between giving you a money order, or simply a copy of a money order. In the former case you are free to set the payee name to be whatever you want and cash/spend that money order. In the latter, you can do whatever you want to your copy, but it doesn't affect mine.</p> <p><abbr title="value-type object">VTO</abbr>s automatically enforce the making of copies, you simply cannot change the original, no matter what... though you might change the property values on your copy, this does nothing to my original properties. What this means is that comparing value-type objects cannot meaningfully compare the <strong>physical</strong> identity (e.g. the reference handle/memory address) between to value-type objects because they will <strong>always</strong> be different.</p> <p>So, how do you meaningfully compare <abbr title="value-type object">VTO</abbr>s? By their <strong>logical</strong> identity. In the example of a money order, the <strong>logical</strong> identity is actually the money order number, not the physical piece of paper. Some less-sophisticated verifications of the money order's validity might hinge on the appearance of the piece of paper, but a much better authoritative verification comes from calling in the money order number to the issuer and seeing if that number is still valid and for what amount. Even modern sporting event venues operate similarly, checking not the physical appearance of of a ticket; rather they scan the barcode and match that against a database to insure the ticket is valid and hasn't been used yet.</p> <p>Thus, a <abbr title="value-type object">VTO</abbr>'s identity must be defined in terms of one or more of the property values. To check <strong>logical</strong> equality of two <abbr title="value-type object">VTO</abbr>s, you compare the equality of the <em>identifying</em> properties. In the case of a money order, the money order number.</p> <h3>Collections and hash-codes</h3> <p>When you drop an object in collection you expect to be able to later be able to retrieve that object (or, in the case of a <abbr title="value-type object">VTO</abbr>, a copy of the object) back out. The simplest way is enumeration, but that's not very quick. More commonly, you stick the object in some sort of dictionary keyed by some value. In the case of an <code>Array</code>, the key is simply the integer index of where you stuck the item, but for large numbers of potential objects you really need a <em>identifying</em> property on the object itself. In the event ticket example, it's the barcode of the ticket. That key is used to store and retrieve the ticket information into a collection (perhaps a <code>Dictionary<TicketNumber, TicketStatus></code> collection) is the barcode value. To make the storage and lookup quick, the collection internally stores the key values in "buckets" that are based in some way on the key's value. Each "bucket" contains a list of objects that have the same key-gives-bucket-number collection. Once you find the right bucket, you scan through all the objects in that bucket by doing an identity comparison. This means that:</p> <ol> <li>There must be some way to map the key values into bucket numbers. <li>The mapping should not change if the identity of the object doesn't change. <li>When two objects map to the same bucket number, they are disambiguated using the identity comparison. <li>That once you've placed an reference-type object in a collection, changes to it's <em>identifying</em> properties are going to break the comparison on retrieval. <li>Changes to a value-type-object are impossible, since the collection is holding a copy.</li></ol> <p>The standard method used in the .Net Framework Class Library for identity comparison is the the <code>Equals()</code> method. The standard method in the FCL to map object key values into buckets is the <code>GetHashCode()</code> method. In practical terms, this means that the <code>Equals()</code> method and the <code>GetHashCode()</code> method work together for any object you might want to place in a collection. They must agree one what properties of an object are <em>identifying</em>. When you design a value-type object, you really have to get it right because they have no meaningful <strong>physical</strong> identity.</p> <h3><code>Equals()</code> and <code>GetHashCode()</code> are free!</h3> <p>In the the .Net runtime, all objects automatically inherit an implementation of both the <code>Equals()</code> method and the <code>GetHashCode()</code> method. But as with many things in life, not all free things are really worth much. The default implementation of the <code>Equals()</code> method for reference-type objects is simply to compare the reference handle for equality. If we're pointing at the same object, we're talking about equal objects. The default implementation of the <code>GetHashCode()</code> method similarly bases its answer on the reference handle value. For value-type objects, the .Net runtime treats them <em>as-if</em> they inherit from <code>ValueType</code>, so the the <code>Equals()</code> method on <code>ValueType</code> is what is called. This method compares field-by-field the individual elements of the object and returns if each is equal. Likewise, the the <code>GetHashCode()</code> method of <code>ValueType</code> is the default and it merely computes and combines the field-by-field hash-codes and combines them in an unspecified way to generate an overall object hash-code.</p> <p>In summary, this means is that the default treatment of <abbr title="value-type object">VTO</abbr>s is to treat <strong>all </strong>fields as identifying. The default treatment of <abbr title="reference-type object">RTO</abbr>s is to treat <strong>none </strong>of the fields as identifying. Rarely would this be the right thing to do, but that's what you get for the low-low-price of free.</p> <p>If you have a <strong>logical </strong>identity for a <abbr title="value-type object">VTO,</abbr> or an <abbr title="reference-type object">RTO.</abbr> then you need to supply your own implementation of <code>Equals()</code> and <code>GetHashCode()</code>. As detailed above, you need to make sure that they are coupled in their understanding of what fields and/or properties are the <em>identifying </em>ones.</p> <h3>Building your equality</h3> <p>Once you've identified what fields or properties to use when comparing to objects for <strong>logical</strong> identity, you need to implement an <code>Equals()</code> method and a <code>GetHashCode()</code> method the right way. For the Equals() method, there are <a title="Guidelines for Implementing Equals and the Equality Operator" href="http://msdn2.microsoft.com/en-us/library/7h9bszxx(vs.71).aspx">only a few rules</a>:</p> <ol> <li>Thou shalt <strong>not</strong> throw an exception <li>Thou shalt implement reasonable overrides (at least for your own type and taking <code>System.Object</code>) <li>If you override <code>operator ==</code>, you must have a corresponding <code>Equals()</code> method. <li>If you implement the <code>IComparable</code> interface, you should override <code>Equals()</code></li></ol> <p>So, a classic implementation of a value-type object would be something like this (borrowed from Davy's post):</p><pre>public override bool Equals(object obj)
{
Address address = obj as Address;
if (address != null)
{
return this.Equals(address);
}
return object.Equals(obj);
}
public bool Equals(Address address)
{
if (address != null)
{
return this.Street.Equals(address.Street)
&& this.City.Equals(address.City)
&& this.Region.Equals(address.Region)
&& this.PostalCode.Equals(address.PostalCode)
&& this.Country.Equals(address.Country);
}
return false;
}
</pre>
<p>Note that it's perfectly fine for the Address object to have many other properties that are not considered <em>identifying</em> and thus not included in the implementation of the <code>Equals()</code> method. That's really the whole point of implementing the <code>Equals()</code> method on an object. For <abbr title="value-type object">VTO</abbr>s you are trying to ignore some fields that the default <code>ValueType</code> implementation would have included. For <abbr title="reference-type object">RTO</abbr>s, you are trying to establish some properties that give <strong>logical</strong> equivalence.</p>
<h3>Computing a useful hash-code</h3>
<p>Once you've established a the body for <code>Equals()</code> method, you <em>absolutely must</em> define the <code>GetHashCode()</code> method. This is where Davy's gets it <u>99%</u> right. He correctly states that every field/property value you call <code>Equals()</code> against should also be included in the <code>GetHashCode()</code> return value. Most people get that right, and Davy avoids the common mistake of adding the <code>GetHashCode()</code> sub-values together (which would skew the distribution pattern toward larger absolute values) and does an XOR of the sub-values. This is excellent, but we can get it a tiny bit better by following the pattern of many Microsoft provided classes and shifting the accumulated value before the XOR of the next sub-value. This leads to the low-order bits of the sub-value hash-codes being "distributed" into the final value instead of canceling each other out. Thus, my version of Davy's method is:</p><pre>public override int GetHashCode()
{
return (((((((this.Street.GetHashCode() << 5)
^ this.City.GetHashCode()) << 5)
^ this.Region.GetHashCode()) << 5)
^ this.PostalCode.GetHashCode()) << 5)
^ this.Country.GetHashCode();
}</pre>
<p>Unfortunately, that's kind of ugly and error prone due to all the operator precedence issues. Can we make it better?</p>
<h3>Introducing CombineHashCode</h3>
<p>So, a much better approach would be to have a little helper method set that knows how to do the combining according to this rule. For simplicity and ultimate flexibility, we'll have a version that takes an <code>params</code> array of objects and calls <code>GetHashCode()</code> on each of them in-turn. For better performance (to avoid boxing and unboxing) we'll add a version that takes a <code>params</code> array of precomputed hash codes (actually <code>System.Int32</code> values). Finally, for ultimate performance, we'll have a few overloads that take a specific number of objects or hash-code values to avoid the allocation of the <code>params</code> array. You can add more as needed, but your really ought to rethink your class if you get more than five <em>identifying</em> fields/properties.</p><pre>public static partial class Utilities
{
public static int CombineHashCodes(params int[] hashes)
{
int hash = 0;
for (int index = 0; index < hashes.Length; index++)
{
hash <<= 5;
hash ^= hashes[index];
}
return hash;
}
public static int CombineHashCodes(params object[] objects)
{
int hash = 0;
for (int index = 0; index < objects.Length; index++)
{
int entryHash = 0x61E04917; // slurped from .Net runtime internals...
object entry = objects[index];
if (entry != null)
{
object[] subObjects = entry as object[];
if (subObjects != null)
{
entryHash = Utilities.CombineHashCodes(subObjects);
}
else
{
entryHash = entry.GetHashCode();
}
}
hash <<= 5;
hash ^= entryHash;
}
return hash;
}
public static int CombineHashCodes(int hash1, int hash2)
{
return (hash1 << 5)
^ hash2;
}
public static int CombineHashCodes(int hash1, int hash2, int hash3)
{
return (((hash1 << 5)
^ hash2) << 5)
^ hash3;
}
public static int CombineHashCodes(int hash1, int hash2, int hash3, int hash4)
{
return (((((hash1 << 5)
^ hash2) << 5)
^ hash3) << 5)
^ hash4;
}
public static int CombineHashCodes(int hash1, int hash2, int hash3, int hash4, int hash5)
{
return (((((((hash1 << 5)
^ hash2) << 5)
^ hash3) << 5)
^ hash4) << 5)
^ hash5;
}
public static int CombineHashCodes(object object1, object object2)
{
return CombineHashCodes(object1.GetHashCode()
, object2.GetHashCode());
}
public static int CombineHashCodes(object object1, object object2, object object3)
{
return CombineHashCodes(object1.GetHashCode()
, object2.GetHashCode()
, object3.GetHashCode());
}
public static int CombineHashCodes(object object1, object object2, object object3, object object4)
{
return CombineHashCodes(object1.GetHashCode()
, object2.GetHashCode()
, object3.GetHashCode()
, object4.GetHashCode());
}
}</pre><p>This leaves us with the final version of Davy's <code>GetHashCode()</code> method looking like this:</p><pre>public override int GetHashCode()
{
return CombineHashCodes(this.Street, this.City, this.Region, this.PostalCode, this.Country);
}</pre><p>That's pretty clean and easy to understand, right?</p>
<p><strong>UPDATED</strong>:On 9 March, 2008, I fixed some issues with this posting due to comments from Hugh Brown, make <strong>sure</strong> you use the read the follow-up post <a href="http://musingmarc.blogspot.com/2008/03/sometimes-you-make-hash-of-things.html" title="Oops...">Sometimes you make a hash of things</a>.</p>
<div class="wlWriterSmartContent" id="0767317B-992E-4b12-91E0-4F059A8CECA8:1dc31fa6-5208-48e8-bb68-161ee36cad99" contenteditable="false" style="padding-right: 0px; display: inline; padding-left: 0px; padding-bottom: 0px; margin: 0px; padding-top: 0px">Technorati Tags: <a href="http://technorati.com/tags/C#" rel="tag">C#</a>, <a href="http://technorati.com/tags/.Net" rel="tag">.Net</a>, <a href="http://technorati.com/tags/CLR" rel="tag">CLR</a>, <a href="http://technorati.com/tags/GetHashCode" rel="tag">GetHashCode</a>, <a href="http://technorati.com/tags/Equals" rel="tag">Equals</a></div>IDisposablehttp://www.blogger.com/profile/02275315449689041289noreply@blogger.com5tag:blogger.com,1999:blog-5822946.post-42965580073839943392007-08-10T13:47:00.003-05:002008-04-10T18:49:40.712-05:00Please tell me the rest of the code doesn't look like this...<p>This is <strong>not</strong> right:</p><pre>internal static bool DoesDbExist(SqlConnection conn, string database)<br>{<br> using (SqlCommand cmd = conn.CreateCommand())<br> {<br> <strong>// prefer this to a where clause as this is not prone to injection attacks<br></strong> cmd.CommandText = "SELECT name FROM sys.databases";<br> cmd.CommandType = CommandType.Text;
<p> using (SqlDataReader reader = cmd.ExecuteReader())<br> {<br> while (reader.Read())<br> {<br> string dbName = reader.GetString(0);<br> if (string.Compare(dbName, database, true, CultureInfo.CurrentCulture) == 0)<br> {<br> // the database already exists - return<br> return true;<br> }<br> }<br> }<br> }
<p> return false;<br>}</p></pre>
<p>This <strong>is</strong> right:</p><pre>internal static bool DoesDbExist(SqlConnection conn, string database)<br>{<br> using (SqlCommand cmd = conn.CreateCommand())<br> {<br> cmd.CommandText = "SELECT name FROM sys.databases WHERE name=@name";<br> cmd.CommandType = CommandType.Text;<br> cmd.Parameters.Add(new SqlParameter("@name", database));
<p> using (SqlDataReader reader = cmd.ExecuteReader())<br> {<br> return reader.Read();<br> }<br> }<br>}</pre>
<p>Someone please assure me that this is not how everyone else handles avoiding SQL injection.</p>IDisposablehttp://www.blogger.com/profile/02275315449689041289noreply@blogger.com2tag:blogger.com,1999:blog-5822946.post-60155203598597806232007-06-20T18:29:00.001-05:002007-06-20T18:29:55.140-05:00News flash! Kids grow up doing the things they did as a child.<p>Today, in blinding science, it turns out that kids who "smoke" candy cigarettes <a title="Who would have guessed" href="http://www.parentdish.com/2007/06/19/candy-cigarettes-lead-to-the-real-thing/">are more likely to try the real thing later</a>. Shocking, huh?</p> <p>Can I <strong>please </strong>have my tax dollars back on this one?</p> <div class="wlWriterSmartContent" id="0767317B-992E-4b12-91E0-4F059A8CECA8:0ffe9b5a-d0fa-4381-951c-fb4629338a65" contenteditable="false" style="padding-right: 0px; display: inline; padding-left: 0px; padding-bottom: 0px; margin: 0px; padding-top: 0px">Technorati Tags: <a href="http://technorati.com/tags/blinding%20science" rel="tag">blinding science</a></div>IDisposablehttp://www.blogger.com/profile/02275315449689041289noreply@blogger.com0