DevelopmentNow Blog
 Wednesday, July 18, 2007

SubSonic is an open source project loosely modeled after Rails. It uses BuildProviders to automatically generate the DAL/ORM code at compile time, meaning you don't have to manually regenerate code every time your database schema changes.

The downside was that the code was only auto-generated for Web Site projects, not Web Application or Class Library projects. So Rob Conery has a recent post about using Pre-build Steps to autogenerate the SubSonic code for all types of projects.

So, read the "What will it do for me?" on the SubSonic home page, and check out the first 5-10 minutes of a recent screencast. If you like what you see, maybe try it out in your projects.

If you aren't already using DALs and code generation to accelerate your development work, you really owe it to yourself to check it out. :)

July 18, 2007    Bookmark to Digg or other social bookmarking
#    Disclaimer  |  Comments [0]



 Friday, May 04, 2007
Dan North has a nice intro to BDD over on his site. BDD = Behavior Driven Development, an evolution of TDD (Test Driven Development). He also has some other blog posts about BDD that help illustrate the difference between BDD and TDD and explain why BDD isn't just TDD with prettier names.
May 4, 2007    Bookmark to Digg or other social bookmarking
#    Disclaimer  |  Comments [0]



 Wednesday, February 28, 2007

One of my projects involves a complex financial database (stored in Access). I wanted to add a feature to the client EXE that allowed testers to FTP their current copy of the database to me so that I could try to reproduce problems without having to reenter all the data. I didn't see an FTP control handy in VS2003, and I didn't feel like messing around with Windows built-in FTP command line program, which doesn't support passive mode & is annoying to pass parameters to.

So I found NcFTP, a free command line FTP program that fit the bill. Specifically, it offers a dedicated "uploader" program called NcFTPPut. Now, the codebehind for my "Upload Database" button looks like this

Private Sub UploadDatabase()
  If MessageBox.Show("Upload current database for testing? You must be connected to the internet.", _
      "Confirm Upload", MessageBoxButtons.YesNo) = DialogResult.Yes Then

    ' make temporary copy of database
    Dim currentDB As String = AppConfig.GetDBFileName
    Dim tmpDBName As String = currentDB.Replace(".mdb", "-" & _
      System.DateTime.Now.ToString("M-d-yyyy-h-m") & ".mdb")

    System.IO.File.Copy(currentDB, tmpDBName, True)

    ' upload using NcFTPPut
    Dim exe As String = System.IO.Path.GetDirectoryName(Application.ExecutablePath) & _
      "\ncftpput.exe"
    Dim params As String = "-u MyFTPUserName -p FTPPassword ftp.developmentnow.com . """ & tmpDBName & """"
    Try
      Dim myProcess As Process = System.Diagnostics.Process.Start(exe, params)

      myProcess.WaitForExit()

    Catch ex As Exception
      MessageBox.Show(ex.Message, "Error", MessageBoxButtons.OK, MessageBoxIcon.Error)

    Finally
      ' delete temp copy
      System.IO.File.Delete(tmpDBName)
    End Try
  End If
End Sub

It's simple, but it works. Now whenever testers find problems with the program, they can just click the "Upload Database" button in my app to send me the data I need to reproduce the problem.

 

February 28, 2007    Bookmark to Digg or other social bookmarking
#    Disclaimer  |  Comments [0]



 Friday, February 16, 2007

New version of FireBug is out (v1.0.1), if you use javascript and FireFox, get it! Release notes for 1.0.1:

  1. Support escaping the % sign in console.log() calls using %%
  2. Support "Find Next" in the CSS tab
  3. Fixed problem causing inconsistent breakpoint triggering
  4. Deleting a disabled CSS property will work properly
  5. Fixed bug that prevented editing of DOM properties with numeric values
  6. Inserted warning about incompatibility with Sothink SWF Catcher extension
  7. Fixed incompatibility with HTML Validator extension

And no, I'm not going to post a blog every time a minor rev of something comes out. Only sometimes. :)

Code | Tools
February 16, 2007    Bookmark to Digg or other social bookmarking
#    Disclaimer  |  Comments [0]



FYI, TinyMCE is a nice little editor that makes it super easy to turn a regular textbox into a WYSIWYG HTML editor.

You can play with a demo and see how to install it.

I haven't looked into how to extend it, but if you have a number of textboxes containing editable HTML, you owe it to yourself & your users to bolt something on like TinyMCE.

FWIW I've also worked extensively with FreeTextBox, and while it's nice, installation is a bit more involved, and progress on it seems to have slowed. Plus it's not totally free open source, and the HTML output isn't very XHTML (unless you buy the non-free version).

TinyMCE is cool because it's a literal bolt-on that you can include w/o touching your ASP/ASP.NET/PHP code if you don't want to.

To be fair, another really popular WYGIWYG editor is FCKeditor. I haven't used it, but I've heard it's the bees knees.

And lastly, there's a new potentially-hip WYSIWYM (What You See Is What You Mean) editor cll WYMEditor that I mentioned a while back. Their goal is to keep the outputted XHTML simple, since oftentimes users can paste or SHIFT-click their way to producing crappy HTML that "looks" good to the eye but is horrible (e.g. extra breaks, inline styles) for CMS systems.
Code | Tools | Web
February 16, 2007    Bookmark to Digg or other social bookmarking
#    Disclaimer  |  Comments [0]



I've been playing more with jQuery lately and enjoying it. It's small, quick, cross-browser, CSS-compliant, and won't conflict with any other javascript libraries (including ASP.NET AJAX). There are also a number of easy-to-use plugins that offer some nice almost-turnkey functionality w/ a few lines of javascript (e.g. tabs!).

There are other powerful javascript libraries, like prototype, script.aculo.us (which uses prototype), dojo, mootools, and a bunch of others.

It's tough keeping on top of everything in web development, since there are new tools and widgets all the time. Spend too much time checking out what's new and you never learn anything in depth or get anything built. Spend too little time learning what's new, and you miss out on huge timesavers and risk getting outmanuevered by speedy web 2.x competitors. So I try to maintain a balance, and drink a lot of coffee. :)
Code | Web
February 16, 2007    Bookmark to Digg or other social bookmarking
#    Disclaimer  |  Comments [0]



 Tuesday, February 13, 2007

One day, a friend asks you if you could help them make a simple web site. Your friend said it would be great if it could include things like a blog, photo gallery, maybe an online calendar. Maybe your friend's in a band and wants to have a mailing list and an online calendar. Or they're a budding artist and want a photo gallery to show off their work. Pretty understandable requests, and pretty common features (nowadays).

However, you have a dilemma. Your friend doesn't have a ton of cash, and you don't have a ton of time. While you could build all those features from scratch, that could take a while. You want to make sure that you don't undertake a project that could spiral from hours into days of work. But you don't want to send your friend packing, and there's got to be a lot of free or almost-free stuff out there that you can stick together, right?

Yep, there sure is. Here are some things that I found or used before:

  • Use Google Apps for Domains. It provides everything a basic site could need -- email, calendar, a page editor, and some attractive (albeit a bit generic) page templates allowing someone to build a site in a WYSIWYG way. I didn't see a way to upload a custom page template, but you can edit the HTML of the page "sections" in order to embed stuff like YouTube videos or whatever. I like WYSIWYG editors because it means that you can make your friend work on & enhance the site, instead of you having to make every tweak. Note that you can use Google Apps just for email and calendar -- you don't have to use it for web pages if you don't want to.
  • Use a cheap web host. The cheapest I've found is E-rice, but it's very no-frills. GoDaddy has basic plans for $3+/mo that include PHP, MySQL, email, and a number of ready-to-install apps (blogs, forums, CMSes, photo galleries, etc.) for Linux or Windows hosting plans (Linux has way more freebies). Dreamhost is a bit more expensive and also offers one-click installs for blogs, CMSes, wikis, photo galleries, etc. The free add-on apps available for GoDaddy and Dreamhost accounts are nice, because if you want to include those features later on, you don't have to mess with installation. If you're trying to decide between Linux and Windows, I'd suggest going with Linux/PHP plans, since there's a lot of free/open source PHP code out there that you could include in the site later. And PHP is fun. :) 
  • Use a blogging service like Blogger or Wordpress or Typepad. They offer lots of templates (althoguh you can make your own), you can have them on your own domain, they often offer embeddable widgets, and they're cheap or free. Great for a basic personal or family site. Plus, a blog-driven site means that your friend is in control of the content, which is a good thing. :)
  • If you don't want to use a pre-existing blog template, you can use open source web templates from oswd or OpenWebDesign. Some require attribution, but all are free. Andreas Viklund also linked to some open source templates here and here.
  • You can include free stock art from stock.xchng or flickr. For stock.xchng, all the stock art is free, although some photos require you get the author's permission and/or provide attribution. Note that stock.xchng often includes non-free samples from Stockxpert.com in the search results, so be careful where you click. For "free" stock art on flickr: 1) do a search, 2) click "advanced search", 3) check "only search within creative commons-licensed photos", and 4) re-perform the search. You'll see photos on flickr available under the Creative Commons license, which means you must try to obtain permission and provide attribution (e.g. a link somewhere on your site saying "city photos courtesy of Mike Smith"). There's also a "stock" group in flickr where you can find photos.
  • For photo galleries, you can include widgets & links from flickr. Photobucket has a nice photo album widget that makes it easy to add a photo album to your site. Or you could pay $40-60/year for a clicker, more integrated photo gallery from SmugMug. Or use any number of open source photo galleries (DreamHost & GoDaddy offer one-click installs if you have Linux hosting).
  • You can use Google Calendar to track and share online events (good for band, bar, etc web sites), and embed it into your site. If you're using Google Apps, embedding it is as easy and clicking the "add widget" link. Otherwise you have to go through a few more steps, but it's still easy. 30 boxes is another very nice online calendar, and they have a nice "Share" button in the upper left that makes embedding your calendar in your web site a snap (click "Add to Blog"). Nitpick to 30 boxes: maybe rename that link from "Add to Blog" to "Add to Blog / Web Site" ?
  • You can run a simple mailing list/discussion group using Google Groups or Yahoo Groups. Google Groups has added some neat new features lately.
  • You can find other easy-to-embed widgets (e.g. local weather, latest news, games, a clock(?) )for your web site at WidgetBox or Google Gadgets. Like this weather thingy:

So there's the high level bullet point deal. Now you can help your friends and neighbors put together some basic sites while minimizing the risk of it becoming an ongoing project.

Code | Web
February 13, 2007    Bookmark to Digg or other social bookmarking
#    Disclaimer  |  Comments [0]



 Tuesday, February 06, 2007

Today, TechCrunch posted a review of AllTheCode, a new code search site. So now we have at least

  • AllTheCode  -- only java results right now, no filtering available
  • Krugle -- can specify language and project. Can filter results by where the search term appears (e.g. comments, method name, etc). Can also search "Tech Pages" (a selective web site search) and projects.
  • Koders -- can specify language and license
  • Google Code Search -- can specify language, license, package, & file. Can search using regular expressions.
  • Codase -- can specify language (Java, C, or C++ only), plus advanced things like search within method name, class name, etc.

Why Use Code Search Tools?

Code search tools are handy because as a developer, your productivity and quality can be greatly affected by how you come up with the code to put into your applications. Sure, you could write everything from scratch, but that's too slow. Reuse is a good thing. Ideally you should be able to find existing samples, snippets, libraries, and frameworks that you can adjust and glue together.

In the old days you'd have to store away your favorite code snippets on your PC, or thumb through archived issues of Doctor Dobb's Journal. Later, you'd buy CDROMs filled with sample code, and tap your fingers while your drive whirred away. Eventually those code libraries could be installed on your PC. Recently, you'd use Google and other search engines (well, mostly Google) to search for code. But searching on "VB MD5 Algorithm" might get you some relevant results, but also a lot of junk.

So now we have code-specific search engines, allowing us to (hopefully) enter in our search terms, our programming language, maybe our license preference, and get back a set of results sorted by popularity, maybe with a Digg-esque rating and comment system so we know which code snippets are good and which suck. Right now if I pulled up an MD5 algorithm, how would I know if it even worked correctly?

Testing the Sites

To test the code search engines out, I did a search for "luhn" in Java on all five sites. FYI the Luhn Algorithm is used to test for invalid credit card numbers -- it's nice because a lot of invalid numbers can be eliminated without having to connect to and pass the number to a credit card gateway. All five sites returned matches (Codase with some extra work), and all had an integrated source viewer so I could review the match.

So here's what I found:

AllTheCode

AllTheCode returned 29 matches, but I didn't see any search hit highlighting or anything. A lot of them were duplicates, too.
allthecode

The source code viewer was ug-ly, bad color choices. And interestingly, my term "luhn" wasn't in the document at all. I tried some other searches & it seems AllTheCode is having some sort of issue, because most or all of the matches were totally irrelevant to my search. Chalk it up to Alpha status code, maybe. Hmm...maybe code search engines should let us filter against alpha, beta, or release code? There was a link to download the code file, but no links for the project home page or project zip file. No links to view other files in the project, either.
allthecodeviewer

 

Google Code Search

Google Code Search found about 100 matches and highlighted the search term. It also showed the code license and a link to the zip file containing the code file. It seemed to do a good job at suppressing duplicate files (e.g. the same code file in multiple projects), something none of the other search engines did. Not a huge deal, but nice.
GoogleCodeSearch

The code viewer was spartan (as with many things Google), but it had links to other files in the project, and it automatically scrolled me to the spot in the code where "luhn" was found. A very nice touch, especially for huge files. There were handy links to download either just the code file, or the whole project in a zip file. No link to visit the project's home page (e.g. on SourceForge) though, which might make it annoying if you wanted to check for later versions or supporting documentation.
googleviewer

 

Koders

Koders found 23 matches & showed a bit more information than Google did, like LOC, copyright and/or license info, links to SourceForge project pages, etc. Interestingly, koders runs on ASP.NET, a rarity in the Web 2.0 world.
koders

The code viewer was a bit nicer than Google's, and the search term was highlighted. On the left side was links to other files in the project plus anchor links to the various methods in the code file, which was cool. I still had to scroll or browser search to find out exactly where in the file it said "luhn," but at least it was highlighted. I think they should have provided anchor links to exactly where in the file the term was found. I could download the code file, or browse to a "project home page" of sorts (which had links to SourceForge and some stats), but I didn't see an easy way to download the entire project right from koders. Koders also offers a plugin for IDEs like Vistual Studio and Eclipse.
kodersviewer

koders "project home page"
kodersproject

 

Krugle

Krugle found 21 matches. Like Koders, it had some links to the SourceForge project pages and showed the license. It also has a neat feature allowing you to filter results based on where the search term appears (e.g. only show results where "luhn" is part of the function definition), but this feature didn't work very reliably.
krugle

I must say that Krugle has the prettiest code viewer. I liked the highlighting (though using italics for comments is probably a bad idea for readability), it had a cool treeview of the code repository, and it opened the code files in separate tabs so you could browse through multiple results. Yes I know you can also middle-click a link to open a new browser window tab, but still. I could download the code file and link to the project home page, but I couldn't download the whole project from Krugle.

One bad thing is that Krugle heavily uses AJAX, and thus your browser only shows the www.krugle.com link, no matter where you are. Makes it hard to bookmark stuff. You can click the "Create Link" button to get a greybox popup with a unique tinyurl-ish link to your search results, but that's one extra step I'd rather not take. Another bad thing is it didn't highlight my search term in the code, and my browser's search feature didn't work on the AJAX-y page. So I had to manually scan the whole code file to find out where "luhn" was. Not good for huge code files. :)
krugleviewer

 

Codase

Edit: I liked the idea of offering different types of searches, but got inconsistent results on Codase. A smart query (the default type of query) for "luhn" in java returned no matches, while a free text query for "luhn" in java returned 1. A free text query for "luhn" in all languages returned 11 matches, but Codase said they were all java file. So why those 10 extra matches didn't show up before isn't quite clear to me. FYI, searching for "validate credit card" returned more matches, but it still trailed the other engines. The presentation is nice, with syntax coloring in the search results for better readability.

codaseresults.png

The code browser is also pretty nice looking, too. Nice highlighting, with links to download the code. No project links, there, though.

codaseviewer.png

Conclusion

Right now I'd put Google at the top in terms of number of searches, duplicate file suppression, and barebones ease of use. Koder is at a close second with a better code viewer, handier project links, and some neat tools like IDE plugins. I could see Koders and Google switching places depending on your preferences.

Krugle was in third place with an attractive UI but average features. Krugle offers some interesting stuff that the other engines don't (MyKrugle for attaching notes and saving documents, code position searches, tech page searches), but IMO they seemed like not a lot of people would use them.

Currently, AllTheCode is just too alpha/buggy to consider, plus the fact that it only searches java limits it greatly. Codase could be interesting with its deep filtering ability and syntax coloring, but I think mostly people would just use the "smart search," and the poor number of results and limited languages hurts Codase.

Other Thoughts

FYI, to simulate a developer who might need to validate credit cards but wouldn't know the name of the algorithm (i.e. "luhn"), I also did a general search for "validate credit card" in java code and got a lot of results (100+) in Krugle, Koders, and Google Code Search. The results returned seemed applicable, and interestingly, the top few matches were different for all three engines.

Also, I'd like to see some community-based value added in, with comments or diggs to help me decide which algorithm to pick or avoid.

 

 

Code | Tools
February 6, 2007    Bookmark to Digg or other social bookmarking
#    Disclaimer  |  Comments [4]



 Wednesday, January 31, 2007

I came across Unfuddle today. It's an ASP model that offers hosting for project management tools, source code, and bug/issue tickets. Plans range from free to $100/month.

I've been looking for a simple bug tracking solution, and while Fuddle may not be the right way to go (for me), it seems like an interesting offering. They use Subversion for source code hosting, and a Rails-built platform for managing projects, tracking to-dos and milestones, handling bugs and feature requests, entering time spent, and keeping a project moving along. Might be a good option if you have a small team of developers and don't want to deal with setting up a source code repository, and you're tired of emailing Excel spreadsheets around.

Note that hosting companies like Dreamhost offer easy Subversion setup, but AFAIK don't offer the other stuff -- issue and project tracking.

CropperCapture[16].Png

January 31, 2007    Bookmark to Digg or other social bookmarking
#    Disclaimer  |  Comments [0]



Ok, I'm not a designer, I'm a developer. So tips that are new to me are often old hat to my designer friends.

Nonetheless, I put tips out in hopes that they'll be as helpful to me as they are to other non-designer developers who are still often called upon to put together a simple site that still looks decent.

So, I was looking into rounded corners today. You know, the cool hip rounded stuff you see on every Web 2.0 site. So I found Smiley Cat's CSS Rounded Corners Roundup, a big list of rounded corners techniques. What I liked is that many of them had online generators, so I could type in some parameters (color, size, roundedness) and be given the exact HTML, CSS, and images I need to pull off my cool rounded corners effect.

For example, I got this from roundedcornr.com:

 

So, I was looking into rounded corners today. You know, the cool hip rounded stuff you see on every Web 2.0 site. So I found Smiley Cat's CSS Rounded Corners Roundup, a big list of rounded corners techniques. What I liked is that many of them had online generators, so I could type in some parameters (color, size, roundedness) and be given the exact HTML, CSS, and images I need to pull off my cool rounded corners effect.

I also ran across a javascript/css-only version called Nifty Corners Cube. Since it only uses CSS and javascript to give rounded corners to DIVs, no images are needed, and the latest version of NiftyCorners lets you do other stuff like rounded menu tabs, or complex layouts.

Code | Web
January 31, 2007    Bookmark to Digg or other social bookmarking
#    Disclaimer  |  Comments [0]



 Tuesday, January 23, 2007

Smashing Magazine came up with a link-list post called 53 CSS Techniques You Couldn't Live Without. It links to cool CSS effects that are easy enough to add to your web applications. Developers (like me) are notorious for being, hmm, spartan when it comes to design, so ready-to-go script that lets me jazz up my pages is welcome. Plus, Smashing included little screenshots of the effects, so you don't have to click through to see what they're like (though a few of the screenshots don't really explain the effect well [I'm talking to you, Link Thumbnail]). Anyhow, here's images of a few:

CSS Teaser Box
CSS Teaser Box

CSS Ratings Selector

 

ASP.NET | Code | Web
January 23, 2007    Bookmark to Digg or other social bookmarking
#    Disclaimer  |  Comments [0]



 Sunday, January 21, 2007

I was reading TechCrunch today and noticed an article about booBox, a lightbox product that allows Amazon affiliates to incorporate cool web-2.0 "popups" with Amazon.com products into their site. I often read about startups on TechCrunch, and sometimes I think to myself "man, I wish I had thought of that" or "man, I could do something like that."

Well, today the barrier to entry was so low that I came up with a competing product in under an hour. And not only that, but I'm offering three times the options! And did I mention it was free? So I give you ... the DevelopmentNow Amazon GreyBox!

For demo and code, go here.

This is a real life example of what I was getting at in my Social Networking for Sale post -- with rapid development techniques, open source software, and the huge availability of turnkey widgets, code samples, and solutions, product development is becoming increasingly commoditized, allowing the easy output of "close enough clones." A previous employer had experience with a competitor whose product was "close enough" to be a real competitive threat, and so winning clients was less about the actual product than the strength of the team, marketing, PR, customer service, existing client list, and sales power.

So do I think that I'll be a serious competitor for booBox? Probably not, unless I put together a hip-looking web site, send out press releases, work the conferences, etc. And competing with them wasn't really the point. Rather, since Mike Arrington gave booBox "an early thumbs up" and said it "may be quick acquisition bait for Amazon or eBay," it seems there's potential gold even for quickly-developed apps.

Granted, I were serious (or smart?), I probably should have said my product took weeks/months to develop, not minutes/hours. And instead of using my product to prove a point on the commoditization of software in a little-read blog, I should have instead used it to go for either some web 2.0 notoriety and/or a quick-hit acquisition. But ah well. :)

booBox

booBox

DevelopmentNow's Amazon GreyBox

product link

mini link

related products

January 21, 2007    Bookmark to Digg or other social bookmarking
#    Disclaimer  |  Comments [0]



 Monday, January 15, 2007

Well, I may have to start up a social networking category. I ran across Shuzak, a "social network for geeks." While the idea of niche-based social networks (or vertical social networks: VSNs) isn't a stunner, it doesn't seem very common yet. I'm sure we'll see many more in the upcoming year.

Shuzak seems interesting, but at first blush it feels less like a targeted MySpace and more like a BBS with extended user profiles. Like a custom Zoints Local app. I do like the fact that they're not just a generic social network, though ... they have a few features specific to their theme (e.g. syntactical code highlighting, mathematical equation formatting). I could maybe see this taking off in a university, allowing professors to set up private, invitation-only "classroom social networks" to allow their students to collaborate on projects, exchange notes, ask questions, etc. There ya go, Shuzak, I just gave you your in. :) Ofc, you probably already thought of that.

Anyhow, when one (like me) talks about a social network being a BBS with better profiles, is that a bad thing? How exactly does one define a social network? Is it features, e.g. profiles, groups, and buddy lists? Or is it purpose, e.g. exchange ideas, send messages, meet people? And if it's purpose, does that purpose have to be intentional (e.g. using eHarmony to find dates), or can it be incidental, as with the countless friendships forged in MMORPGs? Could you argue that other, existing sites that are centered around communication (forums, social bookmarking sites that allow comments) are social networks, too? Mashable sure did by mentioning Digg as a techie social network.

I will say, though, that looking enough "like MySpace or Facebook" will probably make it easier to get funded.

January 15, 2007    Bookmark to Digg or other social bookmarking
#    Disclaimer  |  Comments [0]



 Sunday, January 14, 2007

As I mentioned in Social Networking for Sale, I believe social networking software/sites (SNS) will become pretty commoditized in 2007. It's already pretty cheap & easy to get your own community site.

For example, there's Scuttle, open source social bookmarking software. If you need social networking software (a la MySpace), you can find it for free/open source with Alicia (aka PHPizabi) or AroundMe or osDate or Yogurt or Dolphin.

If you want a fancier MySpace clone you can spend $300 or so for phpFox or Handshakes or BuddyZone or webNetwork or Elgg Spaces or SocialEngine or a dozen others. 

There are also hosted, turnkey solutions like PeopleAggregator, Me.com, NingPringo, KickApps, and others, which offer plans ranging from free to paid.

There's also a social networking addon for vBulletin called Zoints Local -- plug it into your existing vBulletin site and bingo! instant "community."

And of course there are "community" addons for CMS+ platforms such as phpNuke, Joomla (Community Builder), Drupal, etc. allowing you to truly build your own SNS. One could also do it by hand using Rails or some other rapid dev platform.

No matter which option you choose, you have a number of customization options, not all of which require a programmer.

The point of all those links is to reinforce the fact that there's already a slew of cheap starting points for a social community site for would-be MySpace topplers. I figure there will eventually be a number of vertical social networks (VSNs) for gamers, hobbyists, flyfishermen, cheerleaders, etc. Maybe they'll be within MySpace, or maybe third party sites. Better yet would be if VSNs could integrate with people's existing social networks elsewhere on MySpace, Facebook, LinkedIn, etc, so that you don't have to abandon your friends, profile, and blog posts to tap into a more targeted community. People would be more likely to join a new social network if they didn't have to reupload all their photos, reanswer all their profile questions, etc.

PHPizabi Alicia

 

Boonex Dolphin
dolphin.png

OSDate
osdate.png

Zoints Local
Zoints Local

January 14, 2007    Bookmark to Digg or other social bookmarking
#    Disclaimer  |  Comments [2]



 Thursday, January 04, 2007

I've been talking with different clients about building social networking sites. What I'm hearing more is the interest in specialized or vertical social networks (VSNs). Then what I hear is "so, how much would it cost to build a web 2.0 social network for <insert niche here>?" The answer is: it's getting cheaper all the time. The second, less-expected answer? That cheaper-to-build social networking sites isn't necessarily good news for would-be MySpace killers.

Interestingly, I noticed that mashable had a link indicating that ruduzu, the "anti-social networking site", is for sale. The winning bidder gets all the code, the existing community (all 273 of 'em), and one year of hosting. So far the bid is up to $3,800 with 15 days to go, so it's possible that potential MySpace killers can get their very own social site for well under $10,000!

I do think that plug & play social networking features (social widgets?) might be more popular in 2007. TechCrunch already talked about a comment system that could be quickly embedded in any site. I blogged about Plaxo's Address Book Widget making it easy to add all your buddies to a new social web site (and I wouldn't mind being able to import actual buddies from other social networks, too). So I'm sure we'll see other widgets (instant photo gallery! instant blog! instant buddy list!) this year, along with a huge crop of rapidly developed (and probably rapidly abandoned) "web 2.0" sites.

Thus comes my real point -- I think the base technology is becoming more of a commodity. I believe it's getting easier than ever to develop software and web sites, and developers are more reluctant to reinvent the wheel. Which IMO means two things

  • the barrier to entry for crappy "me too!" sites will continue to get lower
  • the differentiators will be (as in the past):
    • continual improvement & innovation
    • ability to raise and manage capital
    • ability to market and make deals
    • ability to serve up interesting content 
    • hard, continued work

So in 2007, if you wanted to knock out a quick & dirty MySpace clone in a few weeks, you probably could. You could make a Google Maps mashup in under a week. Maybe even build a deli.cio.us knockoff in a few days.

But if you want those sites to be something other than resume fodder, expect to put in some hard time. The days of "build it and they will come" are gone. If they ever existed at all.

January 4, 2007    Bookmark to Digg or other social bookmarking
#    Disclaimer  |  Comments [0]



 Monday, December 18, 2006

I saw this nice Code Smell summary at Berkeley thanks to Diederik Krols' anti-pattern blog. The Code Smell Summary (or maybe it's a cheat sheet) gives nice overviews of bad code practices, (briefly) explains why they're bad, and provides links to refactoring.com's illustrations and examples to demonstrate the problems and the cures.

Reviewing the code smell summary will help you become more aware of problem code in your own projects. Then if you really want feedback on your code, you can start using code analysis tools like FxCop or NDepend. :)

You can also read through some of the common refactorings as well as this list of database refactorings, although there's some sifting required to get to the real "a ha, that's a good idea!" stuff. I will say that the refactoring sites don't provide much explanation on why the ideas should be followed, presumably so that we'll be forced to buy the books.

Code | Tools
December 18, 2006    Bookmark to Digg or other social bookmarking
#    Disclaimer  |  Comments [0]



 Friday, December 01, 2006
MIT has a cool timeline widget under open source.

Unfortunately I can't really stick it inline here, but you can visit the link to check it out.
December 1, 2006    Bookmark to Digg or other social bookmarking
#    Disclaimer  |  Comments [0]



 Wednesday, November 29, 2006

Maybe you've seen some of the new cool web sites that automatically let you import people from your Outlook, Yahoo, Gmail, or other contact lists. That way your users don't have to remember or type in their friend's email addresses ... they just pick from an imported list.

Well I noticed that Plaxo has an Address Book Widget you can embed in your site that allows your site visitors to import contacts from Yahoo, etc. It would probably take you under an hour to implement on your site.

widget screenshot

FYI, the widget is free, but it contains a link to Plaxo.com.

ASP.NET | Code | Web
November 29, 2006    Bookmark to Digg or other social bookmarking
#    Disclaimer  |  Comments [0]



 Tuesday, November 28, 2006

Mads Kristensen posted an article about an HttpModule to move ViewState to the bottom of the page. It's based on Scott Hanselman's similar blog post.

The nice thing about moving ViewState to the bottom of the page is spiders don't have to sift through it, which means potentially better spidering and search engine ranking. Another benefit is that your page may render faster on browsers since more of the visible HTML is retrieved sooner.

I like the idea of making an HttpModule out of it, because then you don't have to have a custom Page class or anything .. you can just drop the HttpModule into any project & it starts working. Assuming it doesn't have huge bugs, of course. :)

November 28, 2006    Bookmark to Digg or other social bookmarking
#    Disclaimer  |  Comments [0]



 Tuesday, November 21, 2006

Some good comments at accessify.com about screen reader testing tools:

Some good points were also raised about sighted developers using screen readers for testing

Adrian Higginbotham: Personally I never recommend anyone to try testing a site with a screenreader product unless they are already a confident screenreader user. you simply don’t use the tool in the same way

Adam Perry: I am sure Adrians comments are correct: a Visually Impaired (VI?) person will use the screen reader in a very different way. Nevertheless I found using a screen reader helped to identify areas of HTML which require improvement (inadequate or missing ALT, TITLE and SUMMARY attributes, for instance).

November 21, 2006    Bookmark to Digg or other social bookmarking
#    Disclaimer  |  Comments [0]



When developing a web site for screen readers and other accessible browsers, you often have to wrestle with the demands of Section 508 compliance and a nice design.

One WCAG guideline is to provide a "skip navigation" link near the top of the pages. Since you want this link to be available for screen readers, but hidden for normal browsers, your first instinct would be to do something like this:

<div style="display:none"><a href="#content">skip to main content</a></div>

... a bunch of navigation here ...

<a name="content" />
Main content here ...

However, you'd be wrong. I found this screen reader test at Access Matters that indicates that many screen readers try to obey CSS commands, and so many "hidden CSS tricks" like display:none or a 1x1 blank pixel with ALT text end up not getting read aloud by browsers.

It seems like the best solution, if you need to hide the "skip to content" link, is to use something like this

<div style="position:absolute;top:-100px;"><a href="#content">skip to main content</a></div>

Edit: FYI, Jaws and Window-Eyes are the first & second most popular screen readers, at least in 2003 when they commanded 65 & 35 percent of the market, respectively. But more recent articles imply the same thing -- those are the two to support.

 

November 21, 2006    Bookmark to Digg or other social bookmarking
#    Disclaimer  |  Comments [0]



 Sunday, October 29, 2006

http://ankhsvn.tigris.org/screenshots.htmlRC4 of AnkhSVN, a Visual Studio plugin that works with Subversion, was released yesterday. From the site:

AnkhSVN is a Visual Studio .NET addin for the Subversion version control system. It allows you to perform the most common version control operations directly from inside the VS.NET IDE. Not all the functionality provided by SVN is (yet) supported, but the majority of operations that support the daily workflow are implemented.

A screenshot from the screenshots page:

Code | Tools
October 29, 2006    Bookmark to Digg or other social bookmarking
#    Disclaimer  |  Comments [0]



 Wednesday, October 25, 2006

Ok so this is one of my favorite tips in .NET. Of course, there are a lot of things I like about the framework, but anyhow...

A lot of times you'll want to create an array of items, but you don't know how big the array is going to be. You're hoping you can declare an array & just dynamically add items to it like you can with an ArrayList. The first impulse might be to do something like

string[] foo = new string[] { };
foo[1] = "some text";
foo[2] = "some text";

But, that won't work -- you'll get an System.IndexOutOfBoundsException.

However, in .NET 1.1, your friend the ArrayList comes to the rescue with this fun stuff:

ArrayList fooArrayList = new ArrayList();
fooArrayList.Add("some text");
fooArrayList.Add("some more text");

string[] foo = (string[])fooArrayList.ToArray(typeof(string));

Yay! Now you can dynamically add stuff to an array. But wait, there's more -- with .NET 2.0, the Array class somes with a new Resize() generic method, so you can also do this:

string[] foo3 = new string[]{};

// resize array & add an item
Array.Resize<string>(ref foo3, foo3.Length + 1);
foo3[foo3.Length - 1] = "some text";

// resize array & add an item
Array.Resize<string>(ref foo3, foo3.Length + 1);
foo3[foo3.Length - 1] = "some text";

So that's cool. But which is faster, you ask? Let's see, here's the code we used to add 10,000 items to an array.

const int MAX_ITEMS = 100000;
DateTime start;
TimeSpan timeTaken;

// time ArrayList
start = DateTime.Now;
ArrayList fooArrayList = new ArrayList();
for (int i = 0; i < MAX_ITEMS; i++)
{
fooArrayList.Add("some text");
}
string[] foo2 = (string[])fooArrayList.ToArray(typeof(string));
timeTaken = DateTime.Now.Subtract(start);
Console.WriteLine("ArrayList\t" + timeTaken.Milliseconds + " msec\n\n");

// time Array.Resize
start = DateTime.Now;
string[] foo3 = new string[] { };
for (int i = 0; i < MAX_ITEMS; i++)
{
Array.Resize<string>(ref foo3, foo3.Length + 1);
foo3[foo3.Length - 1] = "some text";
}
timeTaken = DateTime.Now.Subtract(start);
Console.WriteLine("Resize\t" + timeTaken.Milliseconds + " msec\n\n");

// prompt for ENTER
Console.WriteLine("Press ENTER.");
Console.ReadLine();

Final result: ArrayList 0 msec, Array.Resize 250 msec. If we go for 100,000 items, it ends up being ArrayList 10 msec, Array.Resize 45000 msec. ;(

So, that's not surprising. The ArrayList probably only resizes things when it needs to and keeps a buffer. So what if we change Array.Resize to resize the Array in chunks of, say, 10,000? And let's go for 1,000,000 items in the array.

const int MAX_ITEMS = 1000000;
DateTime start;
TimeSpan timeTaken;

// time ArrayList
start = DateTime.Now;
ArrayList fooArrayList = new ArrayList();
for (int i = 0; i < MAX_ITEMS; i++)
{
fooArrayList.Add("some text");
}
string[] foo2 = (string[])fooArrayList.ToArray(typeof(string));
timeTaken = DateTime.Now.Subtract(start);
Console.WriteLine("ArrayList\t" + timeTaken.Milliseconds + " msec\n\n");

// time Array.Resize
start = DateTime.Now;
string[] foo3 = new string[] { };
int itemCount = 0;
int arrayChunkSize = 10000;
for (int i = 0; i < MAX_ITEMS; i++)
{
// only resize the array if we need to, and do it in chunks
if (foo3.Length <= itemCount)
Array.Resize<string>(ref foo3, itemCount + arrayChunkSize);
foo3[itemCount] = "some text";
itemCount++;
}
// trim the final size of the array
Array.Resize<string>(ref foo3, itemCount);
timeTaken = DateTime.Now.Subtract(start);
Console.WriteLine("Resize\t" + timeTaken.Milliseconds + " msec\n\n");

// prompt for ENTER
Console.WriteLine("Press ENTER.");
Console.ReadLine();

Much better. ArrayList 150 msec, Array.Resize (chunked) 700 msec. But, ArrayList is still faster. So I guess ArrayList is pretty efficient about expanding its buffer when you add items to it, which is a good thing, since I like ArrayLists.

So...there's your tip, plus we looked at a new Array generic method, and did a bit of performance analysis. If you need a dynamically-sized array, just use an ArrayList and cast it to an array using the .ToArray function. Remember, you can do this for any array, even an array of custom objects.

October 25, 2006    Bookmark to Digg or other social bookmarking
#    Disclaimer  |  Comments [2]



 Friday, October 20, 2006

I've been doing some HTML parsing & cleaning lately, which often involves a lot of regular expressions. Turns out that .* doesn't match across newlines, though, so that if you want to grab an HTML page's title value, the following regular expression:

<title>(.*?)</title>

works for this HTML

<title>this is my page title</title>

but not for this HTML

<title>this is my
page title</title>

I usually use regexlib.com for patterns, tips, & testing, & it didn't say anything about the period . not matching against newlines. It supposedly matches any character, but apparently not CR or LF. So...after banging my head against a wall for a while I finally found a good tip at OsterMiller.org that suggested this pattern

<title>((.|[\r\n])*?)</title>

and voila, it worked! So I was able to write my GetTagValue function like so

public static string GetTagValue(string html, string tag)
{
string pattern = @"<\s*" + tag + @"[^>]*>((.|[\r\n])*?)</\s*" + tag + @"[^>]*>";
Match m = Regex.Match(html, pattern, RegexOptions.IgnoreCase);

if (m == null)
return String.Empty;

if (!m.Success)
return String.Empty;

return m.Groups[1].Value;

}

October 20, 2006    Bookmark to Digg or other social bookmarking
#    Disclaimer  |  Comments [0]



 Wednesday, October 11, 2006

I was having some odd errors sending MemoryStreams as email attachments and I wasn't finding samples on the web that worked (or compiled). I was finally able to figure out that the MemoryStream needs to be open in order for the send to go through, like so:

string from = "ben@foobar.com";
string to = "you@foobar.com";
string subject = "my email";
string body = "this is an email with an attachment.";
MailMessage mail = new MailMessage(from, to, subject, body);
System.IO.MemoryStream memorystream = new System.IO.MemoryStream();
System.IO.StreamWriter streamwriter = new System.IO.StreamWriter(memorystream);
streamwriter.Write("Hello world!");
mail.Attachments.Add(new Attachment(memorystream, "test.txt", System.Net.Mime.MediaTypeNames.Text.Plain));
SmtpClient smtp = new SmtpClient();
smtp.Send(mail);
streamwriter.Close();
streamwriter.Dispose();
memorystream.Dispose();


 

If the Stream is closed you'll get a System.Net.Mail.SmtpException whose InnerException says "Cannot access a closed Stream."


October 11, 2006    Bookmark to Digg or other social bookmarking
#    Disclaimer  |  Comments [2]



 Friday, August 11, 2006

If this were an ordinary post I'd show you a bunch of code illustrating how to send multipart MIME emails using .NET. But yesterday I ran across DotNetOpenMail, an open-source mail component for .NET. And I don't believe in reinventing the wheel too much.

As a reminder, multipart MIME emails allow you to embed multiple content with different MIME types (e.g. HTML and TEXT) into a single email. That way, recipients with HTML-capable email clients will see the HTML version of your email, while older email programs will display the text version.

In .NET 1.1 (which is what I was developing in yesterday), multipart MIME emails aren't really supported, although if System.Net.Mail uses CDO.Message behind the scenes, you'll automatically get a multipart MIME email generated.

So anyhow, I happily found this open-source component & it appears to work fine for my purposes. And so I thought I'd pass along the tip.

August 11, 2006    Bookmark to Digg or other social bookmarking
#    Disclaimer  |  Comments [0]



 Thursday, July 27, 2006

I forgot to include links for some of the libraries I used in my talk at Code Camp:

Atlas -- http://atlas.asp.net

Anthem -- http://www.anthemdotnet.com

Prototype -- http://prototype.conio.net/

script.aculo.us -- http://script.aculo.us/

 

July 27, 2006    Bookmark to Digg or other social bookmarking
#    Disclaimer  |  Comments [0]



 Wednesday, July 26, 2006

I've spent a lot of time lately playing with Atlas & I'm enjoying working with it, despite the huge javascript payload. I wanted to incorporate some script.aculo.us effects into an Atlas page, and noticed this helpful post from huddletogether:



there seems to be a conflict between Scriptmanager and Scriptaculous
you need to place the Scriptaculous after the scriptmanager.
add the following lines of code to get it working:
<body>
<form id="form1" runat="server">
<atlas:ScriptManager ID="sm1" EnablePartialRendering="true" runat="Server" />
<script type="text/javascript" src="/js/prototype.js"></script>
<script type="text/javascript" src="/js/scriptaculous.js?load=effects"></script>
<script type="text/javascript" src="/js/lightbox.js"></script>

other html goes here
<body>


I figured there would probably be javascript conflicts (Atlas already uses prototype's $ syntax). I wonder what else happens when intrepid ASP.NET coders want some fancy effects on their pages?

July 26, 2006    Bookmark to Digg or other social bookmarking
#    Disclaimer  |  Comments [0]



 Monday, July 24, 2006
You can download the slides & code from Code Camp 2006 (Ajax and ASP.NET) here. Note the projects are for VS2005.
July 24, 2006    Bookmark to Digg or other social bookmarking
#    Disclaimer  |  Comments [0]



 Thursday, July 13, 2006

Up to base 36, anyhow. This is a port from old C. I think it works (did it on paper).

// return decimal version of any number up to base 36
// e.g. strtonum("110", 16) returns 272 (which is 110 in base 16)
int strtonum(orig, base)
{
    string digits = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ";
    int retval = 0;

    for (int i = 0; i < orig.length; i++)
    {
        string character = orig[i];

        for (j = 0; j < digits.length; j++)
        {
            if (character == digits[j])
            {
                retval = retval * base + j;
                break;
            }
        }
    }

    return retval;
}

July 13, 2006    Bookmark to Digg or other social bookmarking
#    Disclaimer  |  Comments [0]



 Monday, June 26, 2006

So you have an ASP.NET application and want to do some custom logging other than by using Response.AppendToLog(). Or maybe you have a console app that you want to log what it's doing. Good! Let's get you set up using log4net (a very popular logging library) in a few minutes so you can start keeping track of things. Much of this is based on Haacked's Quick and Dirty Guide to log4net, which is based on VS2003 but still a good read. I also got some tips for Visual Studio 2005 from Akash's blog. Now let's get started.

Download log4net

Download and extract the latest stable release of log4net here.

Add a reference

In your Visual Studio project, add a reference (Project->Add Reference) to log4net.dll (in the bin\net\<dotnet version>\release directory where you extracted the log4net zip file).

Add a log4net config file

Even though you can add log4net configuration settings to your app's main .config file, we're going to give log4net it's own config file for two reasons: 1) cleanliness, and 2) you can log4net change the config settings on the fly without having to restart the app. :)

So, create a new .config file in the root directory of your application ad name it log4net.config. If you're building a client app, set the "Copy to Output Directory" property to "Copy Always". Remove everything in the log4net.config file, and paste the below XML into it.  It contains sample configurations to log many types of events (info, warning, error, fatal) to a rolling set of log files in a "log4net" subdirectory in your application's root directory. It will also send emails for any error or fatal events. You'll at least want to change the Smtp settings to a valid host & email account.

<?xml version="1.0"?>
<log4net>
    <appender name="SmtpAppender" type="log4net.Appender.SmtpAppender">
        <to value="support@yourcompany.com" />
        <from value="support@yourcompany.com" />
        <subject value="ERROR on site" />
        <smtpHost value="your.smtp.host" />
        <bufferSize value="256" />
        <lossy value="true" />
        <evaluator type="log4net.spi.LevelEvaluator">
            <threshold value="ERROR" />
        </evaluator>
        <layout type="log4net.Layout.PatternLayout">
            <conversionPattern value="%-5p %d [ThreadId: %t] Class:%c{1} Method:%M %nMESSAGE:%n%m%n%n" />
        </layout>
    </appender>
    <appender name="RollingLogFileAppender" type="log4net.Appender.RollingFileAppender">
        <file value="log4net\\logfile.txt" />
        <appendToFile value="true" />
        <datePattern value="yyyyMMdd" />
        <rollingStyle value="Date" />
        <filter type="log4net.Filter.LevelRangeFilter">
            <acceptOnMatch value="true" />
            <levelMin value="INFO" />
            <levelMax value="FATAL" />
        </filter>
        <layout type="log4net.Layout.PatternLayout">
            <conversionPattern value="%-5p %d %5rms %-22.22c{1} %-18.18M - %m%n" />
        </layout>
    </appender>
    <root>
        <level value="DEBUG" />
        <appender-ref ref="SmtpAppender" />
        <appender-ref ref="RollingLogFileAppender" />
    </root>
</log4net>

Initialize the configuration

If you're using a web application, add the following line to your Application_Start method in global.asax.

log4net.Config.XmlConfigurator.ConfigureAndWatch(new System.IO.FileInfo(Server.MapPath("log4net.config")));

If you're using a client application (e.g. a console app), add the following line to your app's initialization routine:

log4net.Config.XmlConfigurator.ConfigureAndWatch(new System.IO.FileInfo("log4net.config"));

These lines tell log4net to load configuration information from the local log4net.config file and watch it for any configuration changes while the app is running.

Start logging

Now wherever you want to log some information in a particular class, add a member variable like this

private log4net.ILog logger = log4net.LogManager.GetLogger(System.Reflection.MethodBase.GetCurrentMethod().DeclaringType);

This creates a logger variable, and causes log messages to be prefixed with the class name containing the logger variable. That makes it easier to sift through logs, since you'll be able to see which messages came from which classes.

Then when you want to do some logging you can just use statements like this

logger.Info("some info");
logger.Warn("a stern warning!")
logger.Error("An error occurred!");

Reading the logs

The sample config (above) creates a log file named logfile.txt in a log4net subdirectory inside your application's root directory. Note that for client applications, that'll be your bin/debug or bin/release folders (or wherever the EXE lives). Since we used a RollingLogAppender, older logs will be archived with the date apprended to the file name. That means that "logfile.txt" only contains messages for the current day.

Deploying your app

When you deploy your application, make sure to deploy the log4net.dll, log4net.xml, and log4net.config files. Also make sure that your application's account (e.g. the ASPNET account for web apps, or the user for client apps) can create and write to the "log4net" subdirectory. For web applications you may want to go ahead and create the "log4net" subdirectory up front and assign ASPNET full control rights to that.

Tweaking the configuration

You can tweak the settings in your log4net.config at any time, even while the app is running. You may wish to change the location of the log file, or add different appenders, or change what type of messages get logged. I suggest reading the log4net documentation for more information.

 

 

 

June 26, 2006    Bookmark to Digg or other social bookmarking
#    Disclaimer  |  Comments [0]



 Friday, June 23, 2006

Google recently released an API for Google Calendar. Even though the Google Calendar UI is already pretty slick, developers can now write applications to extend the functionality further. C# and Java versions of the API are available.

For example, you can query your upcoming events in C# like so:

  1. First download & install the libraries. You may need to build them first.
  2. Make a new project and reference GData.dll, GDataExtensions.dll, and CalendarService.dll
  3. Use code kinda like the below :)


using System;
using System.IO;
using System.Xml;
using System.Net;
using Google.GData.Client;
using Google.GData.Extensions;
using Google.GData.Calendar;

namespace ConsoleApplication1
{
    class Class1
    {
        [STAThread]
        static void Main(string[] args)
        {
            string calendarURL = "http://www.google.com/calendar/feeds/default/private/full";
            string appName = "MyCompany-MyApp-v1.0";
            string userName = "you@someisp.com";    // google calendar login
            string password = "somepassword"; // google calendar password

            // create a query object
            // get 10 entries that start between now & the next 10 days
            EventQuery query = new EventQuery();
            query.Uri = new Uri(calendarURL);
            query.NumberToRetrieve = 10;
            query.StartTime = System.DateTime.Now;
            query.EndTime = System.DateTime.Today.AddDays(10);

            // connect to the calendar service
            CalendarService service = new CalendarService(appName);
            service.setUserCredentials(userName, password);

            // execute the query & get the results back
            EventFeed calFeed = service.Query(query);

            // iterate through the results
            foreach (EventEntry feedEntry in calFeed.Entries)
            {
                string eventStartDesc = "n/a";
                string eventWhereDesc = "n/a";
                
                if (feedEntry.Times.Count > 0)
                {
                    When eventStart = feedEntry.Times[0];
                    if (eventStart.AllDay)
                        eventStartDesc = "All day";
                    else if (eventStart.StartTime.Date == System.DateTime.Today.Date)
                        eventStartDesc = "Today at " + eventStart.StartTime.ToShortTimeString();
                    else
                        eventStartDesc = eventStart.StartTime.ToString();
                }

                if (feedEntry.Locations.Count > 0)
                {
                    Where where = feedEntry.Locations[0];
                    if (where.ValueString != null || where.Label != null)
                        eventWhereDesc = where.Rel + " " + where.Label + " " + where.ValueString;
                }

                Console.WriteLine("Event:");
                Console.WriteLine("\tTitle: " + feedEntry.Title.Text);
                Console.WriteLine("\tWhen: " + eventStartDesc);
                Console.WriteLine("\tWhere: " + eventWhereDesc);

            }
        }
    }
}


As you can see, it's pretty easy. Note that you should change the username & password information at the top, and make sure you have some upcoming events in your calendar.

You can also download a sample .NET 1.1 project here.

June 23, 2006    Bookmark to Digg or other social bookmarking
#    Disclaimer  |  Comments [0]



 Monday, June 12, 2006

I recently needed to parse a tab-delimited file and I figured it would be easier if I could manipulate it in DataSet form. I came across Dave O's article on The Code Project which was close, but not quite what I needed. First, I wanted to read the files a line at a time, because some of them are pretty big. Granted, the DataSet will be in-memory the whole time, but I didn't want to allocate all that ram twice -- once for the initial parsing and then again for the DataSet. Second, I needed some extra massaging on the data (e.g. the values are encapsulated in quotes).

So, here's my updated code, which I'll upload to the Code Project one of these days.

using System;
using System.Data;
using System.IO;

namespace AssessmentDownload
{
    class TextToDataSet
    {
        public static DataSet Convert(string fileName,
         string tableName, string delimiter)
        {

            //The DataSet to Return
            DataSet result = new DataSet();

            //Open the file in a stream reader
            StreamReader s = new StreamReader(fileName);

            //Split the first line into the columns       
            string[] columns = s.ReadLine().Split(delimiter.ToCharArray());

            //Add the new DataTable to the RecordSet
            result.Tables.Add(tableName);

            // Cycle the colums, adding those that don't exist yet 
            // and sequencing the one that do.
            for(int columnIndex = 0; columnIndex < columns.Length; columnIndex++)
            {
                string col = columns[columnIndex];
                bool added = false;
                string next = "";
                int i = 0;  // used in case there are duplicate column names
                while (!added)
                {
                    //Build the column name and remove any unwanted characters.
                    string columnname = col + next;
                    columnname = columnname.Replace("#", "");
                    columnname = columnname.Replace("'", "");
                    columnname = columnname.Replace("&", "");
                    columnname = columnname.Trim('\"');

                    //See if the column already exists
                    if (!result.Tables[tableName].Columns.Contains(columnname))
                    {
                        //if it doesn't then we add it here and mark it as added
                        result.Tables[tableName].Columns.Add(columnname);
                        added = true;
                    }
                    else
                    {
                        // if it did exist then we increment the sequencer and try again
                        // with a modified column name (e.g. MyColumn_2)
                        i++;
                        next = "_" + i.ToString();
                    }
                }

                // now update the column array with the newly cleaned column name
                columns[columnIndex] = col;
            }

            // read file a line at a time
            string row;
            while ((row = s.ReadLine()) != null)
            {

                //Split the row at the delimiter.
                string[] items = row.Split(delimiter.ToCharArray());

                // clean up the data a bit (mostly removing surrounding quotes)
                for (int i = 0; i < items.Length; i++)
                {
                    string item = items[i];
                    if (item.StartsWith("'") && item.EndsWith("'"))
                        item = item.Trim('\'');
                    if (item.StartsWith("\"") && item.EndsWith("\""))
                        item = item.Trim('\"');
                    items[i] = item;
                }

                //Add the item
                result.Tables[tableName].Rows.Add(items);
            }


            //Return the imported data.        
            return result;
        }
    }
}
June 12, 2006    Bookmark to Digg or other social bookmarking
#    Disclaimer  |  Comments [0]



 Tuesday, May 23, 2006

I avoided debugging javascript for a long time because 1) I didn't write a lot of it, and 2) it used to be a pain to do so with Visual Studio. Nowadays it's pretty easy, though, if you know how to do it.

You'll notice that if you open up your ASPX page in Visual Studio and try to set a breakpoint in javascript, you'll usually get a "This is not a valid location for a breakpoint" error. What you need to do is open up the client-side version of the page in Visual Studio and debug that. Here's how:

  • Open Internet Explorer, go to Tools->Internet Options, click the Advanced Tab, and ensure that "Disable script debugging" is unchecked.
  • Fire up your web project in Visual Studio and debug it using Debug->Start Debugging (or F5). IE will open up and display your web site.
  • In IE, navigate to the page whose javascript you want to debug
  • In IE, click the View->Script Debugger->Open menu item. That will open up the current page's html and javascript in Visual Studio.
  • Switch back to Visual Studio. You'll see the page's html and javascript. You can now set breakpoints, etc.
  • Now you can switch back to IE and do whatever you want in the page. Any breakpoints that you set in the previous step will be hit, and you can inspect variables, etc. just like in normal code-behind debugging.

If the above annoys you, another javascript-debugging option is to instead use Firefox and Firebug, which I posted about briefly a few months back. Firebug is a powerful plugin that not only lets you debug javascript, but explore the DOM and watch AJAX requests. You can download it here.

Both approaches (Visual Studio and FireBug) have their merits. I suggest trying them both out to see which works for you.

Code | Tools
May 23, 2006    Bookmark to Digg or other social bookmarking
#    Disclaimer  |  Comments [0]



 Friday, May 12, 2006

As I posted previously, this was going to be a longer post. But it got erased because I clicked the wrong button on my mouse, so you all get the Cliffs Notes version. But that's ok, you're a clever group. All three of you. :)

ASP.NET 2.0 Membership

As you may know, ASP.NET 2.0 comes with ASP.NET 2.0 Application Services, a motley gang of controls and APIs that offer basic building blocks for whipping together sophistication web sites with almost no code at all. For example, let's say you wanted to create a web site that required users to register and log in before accessing certain portions, and be able to have different types of users who could access different areas, and even store personalized information about those users. In the past you'd have to code it all yourself, but not any more. There are a series of APIs that you can access and web controls that work against those APIs. For example,

  • Membership -- stores usernames, passwords, and handles credentials
  • Roles -- groups users in roles
  • Profiles -- lets you store specific information (e.g. favorite color) for users
  • Personalization -- works with Web Parts to customize web controls so users can have their own, personalized web pages (think My Yahoo)

One nice thing is the APIs are simply interfaces to the data -- they don't deal with storing it. Instead, data is persisted via various providers that communicate with the APIs. So you could choose to have the Membership API work with a Membership Provider called XMLMembershipProvider that knew how to store membership data in an XML file. You could then change your mind and switch to using a provider called MySQLMembershipProvider that stored data in a MySQL Database, and you wouldn't need to change any code. Think of the APIs as object oriented interfaces, and the providers as implementations of those interfaces.

So Many Providers, So Little Time

ASP.NET comes with a few built-in providers (SQL Server, SQL Server Express, XML), but there's nothing stopping you from writing your own provider if you want to store the data in a different or proprietary way. You can download Microsoft's Provider Toolkit and have a go. As a bonus, there's a sample Microsoft Access provider for those of you who want to/have to store your information in an Access database.

By default, ASP.NET 2.0 uses a set of providers that store data in a SQL Server Express database located in your web site's App_Data folder. Which is nice in that your site is self-contained and you don't need to have "normal" SQL Server. But it's bad in that SQL Server Express is kinda wimply, many hosting providers don't support it, and it complicates xcopy deployment because the last thing you want to do is copy your development usernames on top of the production set.

If you have SQL Server Standard/Express 2000/2005 available, you may as well use it as the provider. Here's how.

Configure Your SQL Server for Application Services

First you'll need to install some stored procedures and tables in whatever SQL Server database you plan on using. Thankfully, there's a tool that makes this easy. Navigate to C:\Windows\Microsoft.NET\Framework\2.0.<latest> and run aspnet_regsql.exe. Perform the following steps:

  • Click Next.
  • Check "Configure SQL Server for application services" and click Next.
  • Connect to your database server and choose your database from the dropdown. The database you choose will have some tables and procs added for membership et al. Click Next.
  • Click Next to confirm.
  • Wait a bit, and you're done. Click Finish.

If you can't or won't run aspnet_regsql.exe (e.g. your company policies require database changes to be in scripts), there are a number of "Install" .sql files that you or your DBA can run. InstallCommon.sql will need to be run first, and you'll need to change the @dbname variable at the top of each script.

Either way, once the database is configured, you can take a look at what got added -- you'll see a number of databases starting with aspnet_ that will contain the membership information.

Add a SQL Server Provider to Your web.config

Next you want to have ASP.NET use that SQL Server database instead of the Express database.

ASP.NET normally accesses the SQL Server Express database via a provider named "LocalSqlServer," and Scott Guthrie posted a tip on renaming that provider so that it instead points to whatever SQL Server database you want. I don't like doing that because a) it's tricking ASP.NET, and b) you run the risk of confusing people by having a provider named "LocalSqlServer" point to something that may not be local. But, it's easy to do if you want to -- just add the following lines to your web.config's <connectionStrings> section:

<remove name="LocalSqlServer" />
<add name="LocalSqlServer" connectionString="YOUR CONNECTION STRING" providerName="System.Data.SqlClient"/>

Anyhow, we're not going to do that. We're going to add explicit lines for our SQL Server database. First ensure there's a connection string in your web.config for your Sql Server.

    <connectionStrings>
        <add name="myConnectionString" connectionString="Data Source=myserver;Initial Catalog=mydatavase;Persist Security Info=True;User ID=myuser;Password=mypassword" providerName="System.Data.SqlClient"/>
    </connectionStrings>

Next, add a Membership section and provider to your web.config's <system.web> section:

<membership>
   <providers>
   <add

      name="MySqlServerMembershipAuthentication"

      connectionStringName="myConnectionString"

      applicationName="/MyApplication"
      description="This is a test database"

      requiresUniqueEmail="false"
      enablePasswordRetrieval="false"

      enablePasswordReset="true"

      requiresQuestionAndAnswer="false"
      passwordFormat="Hashed"

      minRequiredPasswordLength="4"

      minRequiredNonalphanumericCharacters="0"
      type="System.Web.Security.SqlMembershipProvider, System.Web, Version=2.0.3500.0, Culture=neutral,

         PublicKeyToken=b03f5f7f11d50a3a" />
   </providers>
</membership>

Use the right connectionStringName for your database, and give it a name you like. The applicationName attribute doesn't have to be your virtual directory, but it should be unique to your application. This allows different applications to have users with the same name in the same database without colliding. You can read more about the attributes on MSDN, but remember that if you choose Encrypted for passwordFormat, you should include your own validation and decryption keys in your web.config like I blogged about previously on passwordFormat.

Lastly, if you plan on having user roles (Admins, Members, Serfs, etc) add a roleManager section to the <system.web> section:

<roleManager enabled="true">
<providers>
<add name="MySqlServerRoleManagerProvider"
connectionStringName="myConnectionString" applicationName="/MyApplication"
type="System.Web.Security.SqlRoleProvider, System.Web, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" />
</providers>
</roleManager>

Tell ASP.NET to Use Your New Providers

Now let's configure ASP.NET to use the new providers you specified. In Visual Studio, click Website->ASP.NET Configuration from the menu. You'll see the ASP.NET Web Site Administration Tool, which allows you to create users and roles, define application settings, and choose providers. Click the Providers tab, then click "Select a different provider or each feature." You should see your new entries there.

Go ahead and select your new providers, and click Test to make sure they're working correctly. If you want to click the other tabs to see what features they offer, go ahead and do so.

Create a User

Let's create a user, just for testing. Click the Security tab. Click Create User, enter some test values (I named my user "test"), and click the Create User button. It'll say whether the user has been successfully created or not. 

Assuming the user was made successfully, close the window, and take a quick look at your web.config file again. You'll see that the attribute defaultProvider has been added to the roleManager and membership tags. You can actually set that value manually in the web.config instead of using the administration tool.

For fun, connect to your SQL Server database again and view the aspnet_Users table. You should see the new user you created. Notice how the user has a unique UserId and ApplicationId, allowing different applications to have their own separate set of users.

 

If you view the aspnet_Applications table you'll see the applicationName you chose in your membership section. The user's password and other information are stored in the aspnet_Membership table. You can poke around a bit more, then delete the user by opening up the Web Site Administration Tool, going to the Security tab, clicking Manage Users, selecting the user and clicking Delete User.

Roles and Controls

Now that you're set up for ASP.NET Membership, you can open up the Administration Tool again to create some Roles and more users and/or you can go ahead and drag Login, LoginView, ChangePassword, etc. controls onto your web forms. You'll find them under the Login section in the Visual Studio toolbox when editing ASPX pages. Most of the Login controls need no code at all -- you can edit their templates, and they'll automatically read the membership information from your web.config and the providers.

Directories in your site can then be protected by creating a web.config inside them and using simple authorization blocks, e.g. inserting

<?xml version="1.0" encoding="utf-8"?>
<configuration>
<system.web>
<authorization>
<allow roles="SomeRole" />
<deny users="*" />
</authorization>
</system.web>
</configuration>

protects a directory from everyone except users in the role "SomeRole."

Conclusion

Well, what started out as a Cliffs Notes got bigger than I expected, but that's OK. Hopefully I've shown you enough to not only free yourself from SQL Server Express, but encourage some of you to dive into the ASP.NET Membership system.

May 12, 2006    Bookmark to Digg or other social bookmarking
#    Disclaimer  |  Comments [0]



 Thursday, May 04, 2006

I was dealing with an ASP.NET SqlDataSource query that kept timing out, which was annoying because I was trying to run my application through the Database Tuning Advisor (a SQL Server 2005 tool that analyzes trace logs and recommends indexes). The timeouts were preventing me from getting through my whole test case, though.

So, I found a quick solution to setting the timeout value for a SqlDataSource: add a Selecting event handler and put

protected void mySqlDataSource_Selecting(object sender, SqlDataSourceSelectingEventArgs e)
{
e.Command.Command = 300; // 5 min timeout
}

Easy peasy. :)

May 4, 2006    Bookmark to Digg or other social bookmarking
#    Disclaimer  |  Comments [0]



 Monday, April 24, 2006

So I happen to love edit & continue in C# with Visual Studio 2005. There are people who say it encourages sloppy programming (which is true), but for me, there are some things I just can't code as effectively unless I know the runtime state. For example, writing complicated regular expressions against the results of various database queries and algorithms. Do I know if the regular expressions will work exactly before I run the code? No. Do I know if the code will work for all data permutations, beforehand? No.

Hence, edit & continue's blessing. Fire away, hit your asserts or whatnot, and if things look wrong, adjust on the fly. Now, I don't suggest doing big code restructuring while in edit & continue (could get messy & hard to manage), but you are able to if you need to.

Anyhow, much to my dismay I noticed that Edit & Continue stopped working after a while. I couldn't figure out what the deal was, since I was always getting "Changes are not allowed when the debugger has been attached to an already running process or the code being debugged is optimized" whenever I tried to edit the code while debugging. I also noticed that not all my variables were showing up in the Locals & Auto debug windows (this is a hint for the punch line).

Michael Freidgeim has a nice workaround if you get stuck and my simple solution doesn't help you.

His post didn't help, and it took me a few minutes to realize that my project was in "Release" mode instead of "Debug." Whooooops! I guess I didn't notice that because I was able to mostly debug, even in release mode. But I shoulda caught that. Anyhow, flipping back to Debug mode fixed the issue.

Code | Tools
April 24, 2006    Bookmark to Digg or other social bookmarking
#    Disclaimer  |  Comments [0]



 Wednesday, March 01, 2006

ASP.NET 2.0's new membership provider allows for three different ways to protect user's passwords via the passwordFormat attribute:

  • Clear: passwords are stored in clear text. Fine for non-sensitive applications.
  • Encrypted: passwords are encrypted. Note that you will have to put a hard-coded decryption key in the <machineKey> tag in your web.config or machine.config. Otherwise you'll get a "You must specify a non-autogenerated machine key to store passwords in the encrypted format" error when trying to create users. To create a machineKey tag with a set of random tags, you can use my machineKey generator (source code included).
  • Hashed: passwords are not stored in the database at all, only an SHA-1 hash. This means passwords can not be retrieved at all -- if a user forgets their password, they'll have to request a new, randomly-generated one.

Below is an example of a <membership> tag using the Encrypted password format.

        <membership defaultProvider="MySqlMembershipProvider" >
            <providers>
                <add name="MySqlMembershipProvider"
                connectionStringName="MyLocalSQLServer"
                applicationName="MyAppName"
                                 requiresUniqueEmail="false" enablePasswordRetrieval="true"
                                 enablePasswordReset="true" requiresQuestionAndAnswer="false"
                                 passwordFormat="Encrypted"
                                 minRequiredPasswordLength="4"
                                 minRequiredNonalphanumericCharacters="0"
                type="System.Web.Security.SqlMembershipProvider, System.Web, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" />
            </providers>
        </membership>

March 1, 2006    Bookmark to Digg or other social bookmarking
#    Disclaimer  |  Comments [0]



I made some updates to the machineKey generation program I mentioned in my machineKey generation post. It now returns a complete machineKey tag for ASP.NET 1.1 or 2.0 that you can copy and paste into your web.config or machine.config. You can run the generator or download the code here.

March 1, 2006    Bookmark to Digg or other social bookmarking
#    Disclaimer  |  Comments [0]



 Wednesday, February 15, 2006

If you're getting the above error when loading your Visual Studio project, you should first ensure that the web server on your PC is running. Then confirm that .NET is installed by running "aspnet_regiis -i" in your windows\microsoft.net\framework\v1.1.4322 folder. Then open up Control Panel->Administrative Tools->Internet Information Services, and ensure the web site is running.

If you try to start the "Default Web Site" on your PC and get a strange error, make sure you don't have Skype running. It listens on port 80, preventing IIS from doing so. Which prevents your web projects from running in Visual Studio. :)

February 15, 2006    Bookmark to Digg or other social bookmarking
#    Disclaimer  |  Comments [0]



 Tuesday, February 14, 2006

On one of my projects, the client noted getting occasional errors when filling out forms. The errors were something like "Unable to validate data" or HttpException: Invalid_Viewstate. After some research, it turns out the error is due to the viewstate MAC (message authentication code) changing between postbacks, or more specifically, the viewstate MAC in the page is different than the MAC the server is expecting.

What is the ViewState MAC?

MAc stands for Message Authentication Code -- it's a way to ensure that messages haven't been tampered with.

As you know, the ViewState is ASP.NET's way of persisting form data across postbacks (John Peterson has a good overview if you want to know basic ViewState facts). This data appears in base-64 encoded form inside a hidden form value called __VIEWSTATE. The MAC is an additional value designed to prevent changing or tampering with viewstate data. ASP.NET looks at the data in the viewstate and, using a secret validation key, generates a hash from that data using SHA-1. That hash value is the MAC, and ASP.NET includes it in the form. When a postback occurs, your browser passes the form data, the viewstate, and the MAC value to the web server. ASP.NET looks at the submitted viewstate data and generates another MAC value on the fly. It then compares that MAC value to the MAC value contained in the POST information. If the two values match, then no one has tampered with the viewstate. If the newly-generated MAC doesn't match what comes across, however, then ASP.NET believes someone has monkeyed with the data in the viewstate and throws an Invalid_Viewstate error.

Why would the MAC be different from what the server expected?

Several things can cause the MAC to not match what ASP.NET is expecting:

  • Some hacker (maybe even you?) actually tried to change the data in the viewstate
  • The form data got cut off during submission to the server (due to timeout, crappy proxy server, etc)
  • Using Server.Transfer can cause it to happen
  • The validation key used to generate the previous MAC is different than the key being used to generate the new MAC

The last issue can happen more easily that you'd expect. One way is if you're running a web farm. By default, the validation key used to create the MAC is randomly generated by ASP.NET when the application pool starts up. This ensures that the validation key is unique and changes periodically. However, since the key is different from server to server, if you're viewing a page on Server A and post it to Server B, when Server B generates a MAC based on the viewstate data, that value won't match the MAC value when the page was initially served by Server A. Thus, you'll get an Invalid_Viewstate error.

Another reason the validation key (and thus the MAC) will be different is if you cross application pools. If you view a page running in pool A, and post to another page on pool B (e.g. through Server.Transfer), the key will be different & you'll get a mismatch.

Lastly, the validation  key can change mid-session for users if the application pool restarts. Assume some user is viewing a page on your site and filling out a form. While they're doing that, some sysadmin restarts the pool, thus generating a new key. When the user posts the page they'll get an Invalid_Viewstate error. The application can also restart if it is set to shut down while idle (the default is to shut down pools that have been idle for 20 minutes). Imagine a user who views a page on your site, goes away for 10 minutes then maybe spends 20 minutes filling in the form on the page. Meanwhile, no one else is on your site, so the application pool times out & shuts down. When the user finally posts the form (30 minutes after viewing it), the application pool will start back up, create a new validation key, generate a new MAC, notice that the MAC values don't match, and reward you user's diligence with an Invalid_Viewstate error.

Avoiding MAC Mismatch

There are a few ways to avoid MAC mismatch mishaps:

  1. Don't use the ViewState if you don't need to. Not only will it avoid the whole MAC issue, but your pages will run faster to boot.
  2. Turn off MAC generation by setting enableViewStateMac=false in the page or web.config. This isn't recommended, since the MAC helps prevent people from tampering with your viewstate data. But if tampering with viewstate data isn't a concern (and it may not be for some applications where there's no risk of fraud or security breaches), you can turn it off.
  3. Prevent your application pool from restarting by disabling the auto recycle and idle timeout settings in the application pool. This isn't a 100% guarantee that the pool won't restart, but it does help.
  4. Hard-code the MAC validation key so that it's always the same. I recommend this approach for web farms and/or if your application pool keep restarting for whatever reason (overzealous admins, memory leaks, etc). The biggest risk is now your key is hard-coded in a file, so you need to make sure your server is secure so that people don't get that key (otherwise they could hack your viewstate). You can hardcode the key in the <machineKey> tag in the machine.config or web.config, like this:
    <machineKey  
    validationKey="21F090935F6E49C2C797F69BBAAD8402ABD2EE0B667A8B44EA7DD4374267A75D7"           
    decryptionKey="ABAA84D7EC4BB56D75D217CECFFB9628809BDB8BF91CFCD64568A145BE59719F"
    validation="SHA1"
    />

    If you're using ASP.NET 2.0, your machineKey tag should also have the decryption attribute, like this:
    <machineKey  
    validationKey="21F090935F6E49C2C797F69BBAAD8402ABD2EE0B667A8B44EA7DD4374267A75D7"           
    decryptionKey="ABAA84D7EC4BB56D75D217CECFFB9628809BDB8BF91CFCD64568A145BE59719F"
    validation="SHA1"
    decryption="AES"
    />

    Note, the above values are just examples! Generate your own values for validationKey and decryptionKey (and keep them secret) using the code from this MSDN page. A modified version is shown below:

    using System;
    using System.Text;
    using System.Diagnostics;
    using System.Security;
    using System.Security.Cryptography;
    
    class App {
    		static void Main(string[] args)
    		{
    			Debug.WriteLine("64-byte validationKey: " + getRandomKey(64));
    			Debug.WriteLine("24-byte decryptionKey: " + getRandomKey(24));
    		}
    
    		public static string getRandomKey(int bytelength)
    		{
    			int len = bytelength * 2;
    			byte[] buff = new byte[len/2];
    			RNGCryptoServiceProvider rng = new RNGCryptoServiceProvider();
    			rng.GetBytes(buff);
    			StringBuilder sb = new StringBuilder(len);
    			for (int i=0; i<buff.Length; i++)
    				sb.Append(string.Format("{0:X2}", buff[i]));
    			return sb.ToString();
    		}
    }
    

Hopefully the above will help you out if you encounter strange "invalid viewstate" errors, or if you plan to use the viewstate in a web farm, or if you just like knowing more about the viewstate.

P.S. While typing this entry, I accidentally hit the thumb button on my MX510, which by default hits the browser's "Back" button. Which sent me back to the previous page, which made me lose everything I had typed. Which was a super downer. So, disable that mouse button if you don't use it, or you'll be sorry! :)

February 14, 2006    Bookmark to Digg or other social bookmarking
#    Disclaimer  |  Comments [0]



 Tuesday, January 17, 2006

One of my current projects involves creating a data warehouse from multiple datafeeds. For cost reasons we're going with MySQL Community Edition, but we're sticking with ASP.NET, C#, and Windows on the code end.

I'll admit that Windows, C#, and MySQL are an unusual combination, but I love C#, and MySQL recently came out with version 5.0, which features stored procedures, transactions, triggers, foreign keys -- lots of good stuff. How could I resist? If you've never installed MySQL before, don't worry...the installer is pretty easy to follow, and the install guide helps you through. Just make sure you also download the tools and the ODBC connector driver.

Since I'm going with an ELT approach (bulk load into an unconstrained temporary table, perform cleanup & transformations, then import into the "real" tables), I needed a way to review the import process and check the errors. A lot of the examples on the web have you doing everything from the command line, calling the MySQL command line tools from PHP, piping stuff out to text files, etc. I wanted to instead wrap the import process into a nice little C# app. So let's jump into the code!

First we need to connect to the database. I'm using the ODBC driver instead of the new ADO.NET provider, because the ODBC overhead isnt' an issue with so few queries, and the ADO.NET provider is so new that I don't quite trust it yet. :)


string conString = "DRIVER={MySQL ODBC 3.51 Driver}; SERVER=localhost; DATABASE=test; UID=theuser; PASSWORD=thepassword; OPTION=3";
OdbcConnection mysqlConnection = new OdbcConnection(conString);
mysqlConnection.Open();

Next we're going to bump the max error count to something big. The default of 64 just isn't enough when you're importing a lot of data. You want to see all the problems encountered during the import.


OdbcCommand updateErrorCount = new OdbcCommand("set max_error_count = 10000;", mysqlConnection);
updateErrorCount.ExecuteNonQuery();

Now let's import data from a flat file into a staging table using the MySQL LOAD DATA bulk import command. This staging table should have a schema that matches the flat file, but shouldn't have any constraints other than a primary key. If there are any constraints (foreign key, etc), and even one record violates them, then the whole import will roll back. And if you have 1 bad row out of 100,000, you would probably prefer to get the 99,999 good records instead of zero. If you designate a primary key, then you can write a LOAD DATA statement that will only import records whose primary key isn't already in the table. This allows you to import multiple files (or even the same file) repeatedly without worrying about duplicate keys.


string importData =
"LOAD DATA LOCAL INFILE 'F:/imports/mls_import_file.txt' "+
"IGNORE "+
"INTO TABLE mls_staging "+
"FIELDS TERMINATED BY '|' "+
"LINES STARTING BY '' "+
"TERMINATED BY '\r\n' "+
"IGNORE 1 LINES; ";

Let me quickly go over the different lines:

  • LOAD DATA LOCAL INFILE 'F:/imports/mls_import_file.txt' causes a bulk load from the F:/imports/mls_import_file.txt file. The LOCAL keyword has the client program read the file and send it to the server. If you omit the LOCAL keyword, then MySQL will try to read the file directly, which is faster, but may have permissions issues.
  • IGNORE tells MySQL to not import any records whose primary key already exists in the destination table
  • INTO TABLE mls_staging is the destination table. It needs to have the same columns as the flat file, otherwise you need to specify which columns to import.
  • FIELDS TERMINATED BY '|' says that the fields are pipe-delimited
  • LINES STARTING BY '' means the records don't start with any special character. Those are two single-quotes, btw. 
  • TERMINATED BY '\r\n' means each record is terminated by a CRLF
  • IGNORE 1 LINES; tells MySQL to skip the first line (e.g. if it contains column header information). And the semicolon indicates the end of a statement, just like in C#.

Now we're actually execute the import statement:


OdbcCommand importCommand = new OdbcCommand(importData, mysqlConnection);
int importRecords = importCommand.ExecuteNonQuery();
Debug.WriteLine(importRecords + " records imported.");

When the LOAD DATA command runs, it'll pull all those rows and load as many into the table as possible, skipping any duplicates. Since there are no foreign key constraints, the main errors we'll see are if the file can't be read or the server is having problems. However, we might see a number of warnings, e.g. missing columns, too-large field values that get truncated, etc. We want to know about all that stuff, so we're first going to get the number of warnings and errors.


OdbcCommand warningCommand = new OdbcCommand("select @@warning_count;", mysqlConnection);
int warningCount = Convert.ToInt32(warningCommand.ExecuteScalar());
OdbcCommand errorCommand = new OdbcCommand("select @@error_count;", mysqlConnection);
int errorCount = Convert.ToInt32(errorCommand.ExecuteScalar());
Debug.WriteLine(String.Format("{0} warnings, {1} errors", warningCount, errorCount));

Now we can inspect the number of warnings and errors, and if there are any, we can log them, email someone, etc. The below code just loops through the warnings and errors and displays them in the output window.


if (warningCount > 0)
{
// show any warnings
OdbcCommand warningListCommand = new OdbcCommand("show warnings;", mysqlConnection);
OdbcDataReader warningList = warningListCommand.ExecuteReader();
while (warningList.Read())
{
Debug.WriteLine(warningList.GetString(0) + " " + warningList.GetString(1) + " " + warningList.GetString(2));
}
warningList.Close();
}

if (errorCount > 0)
{
// show any errors
OdbcCommand errorListCommand = new OdbcCommand("show errors;", mysqlConnection);
OdbcDataReader errorList = errorListCommand.ExecuteReader();
while (errorList.Read())
{
Debug.WriteLine(errorList.GetString(0) + " " + errorList.GetString(1) + " " + errorList.GetString(2));
}
errorList.Close();
}

And lastly we close up shop by calling

mysqlConnection.Close();

Hopefully the above will help you out when importing data in the unholy(?) alliance of MySQL and C#. MySQL's new version bring along a ton of goodies, so don't be surprised if you see MySQL databases popping up more frequently.

 

 

January 17, 2006    Bookmark to Digg or other social bookmarking
#    Disclaimer  |  Comments [0]



 Friday, January 06, 2006

Sourcegear has an SCM solution called Vault that has a few big things going for it as a VSS alternative:

  • You can import VSS projects
  • It can work alongside VSS, so you don't have to import all your VSS projects
  • It's easy to use and familiar to VSS users
  • It integrates into the Visual Studio IDE (if you're into that)
  • You can connect to it remotely over HTTP or HTTPS
  • It's cheap (free for a single user, $300 per user after that. VSS is ~$500/user)

I'm working on a small project now where the other developer is using Vault for source control, so this gives me a good chance to check it out. I'll also probably use Subversion and VSS 2005 for other projects.

Code | Tools
January 6, 2006    Bookmark to Digg or other social bookmarking
#    Disclaimer  |  Comments [0]



Subversion 1.3 was released a few days ago with a number of enhancements. The biggest news for me is the official support for the Windows _svn hack. In a nutshell, Subversion used to create working directories that started with a dot, but some versions of ASP.NET didn't work with those directories. The "hack" was to start those directories with an underscore instead. You can read more about the hack here. Any utilities will need to call the new Subversion API in order to work with the hack. TortoiseSVN (popular Windows explorer shell plugin for Subversion) has a release candidate that works with the new version -- I'd suggest waiting a week or two for a final version of TortoiseSVN.

What this all means is it will soon be much easier to use Subversion for all .NET development. Which is good, since everyone is trying to dump VSS. :)

* When I say "supports" I mean it officially supports the hack that lets Subversion handle ASP.NET projects and let ASP.NET keep working.

Update: TortoiseSVN 1.3.1 is now released and supports Subversion 1.3.

ASP.NET | Code | Tools
January 6, 2006    Bookmark to Digg or other social bookmarking
#    Disclaimer  |  Comments [0]



 Wednesday, October 19, 2005

It's often handy to have your web site or application send emails. "Thanks for registering." "Thanks for your order." "Hey tech support, the web site is down."

.NET 2.0 provides a few handy classes for outbound emails. Below is some code you can use in your ASP.NET application to send an email. Notice there is some authentication code you can use if your SMTP server requires a username and password in order to send emails. If you aren't sure, try sending an email without a username or password. If you get an SmtpException with a 550 or 553 Relay denied error (SmtpException.StatusCode == SmtpStatusCode.ClientNotPermitted), then you'll need to authenticate. :)

// create the message object
System.Net.Mail.MailMessage myMessage = new System.Net.Mail.MailMessage();

myMessage.To[0] = "some.guy@hotmail.com";
myMessage.From = "some.girl@hotmail.com";
myMessage.Subject = "this is a test message";
myMessage.Body = "hello there!\ntest message!";

// create the SmtpClient object. Specify the SMTP server in the Host property
System.Net.Mail.SmtpClient mySMTPClient = new System.Net.Mail.SmtpClient();
mySMTPClient.Host = "smtpserver.emailfarm.com";

if (SMTP_SERVER_REQUIRES_AUTHENTICATION)
{
	// create the NetworkCredential object with the username & password 
	// to authenticate against the SMTP server
	System.Net.NetworkCredential myCredentials = 
		new System.Net.NetworkCredential("username", "password");
	mySMTPClient.UseDefaultCredentials = false;
	mySMTPClient.Credentials = myCredentials;
}

// send the message
try
{
	mySMTPClient.Send(myMessage);
}
catch (SmtpException e)
{
	Response.Write(e.StatusCode);
}

If you feel the need to email a file to someone, use code like the below

// Attach a file to the email message. 
// The second parameter is the content type (text, binary file, etc.)
System.Net.Mail.Attachment myAttachment = new Attachment(strFilename, MediaTypeNames.Text.Plain);
myMessage.Attachments.Add(myAttachment);

Lastly, you can send an email asynchronously via the SmtpClient.SendAsync(MailMessage, CallbackToken) method. You can attach an event handler to the SmtpClient.SendCompleted event to receive notification when your email has been sent. One big downer is you can only send one asynchronous email at a time -- if you're currently waiting for a previous SendAsync call to finish, other Send or SendAsync calls will fail. Which sortof sucks.

// Send an email without waiting for completion
mySMTPClient.SendCompleted += new SendCompletedEventHandler(MyHandler);
mySMTPClient.SendAsync(myMessage, null);

void MyHandler(System.Object sender, AsyncCompletedEventArgs e)
{
	// code goes here to say "sending complete!" or something
}

Related links:
SmtpClient: http://msdn2.microsoft.com/en-us/library/4971yhhc(en-US,VS.80).aspx
SendAsync: http://msdn2.microsoft.com/en-us/library/x5x13z6h(en-US,VS.80).aspx
Attachment: http://msdn2.microsoft.com/en-us/library/e02kz1ak(en-US,VS.80).aspx

 

October 19, 2005    Bookmark to Digg or other social bookmarking
#    Disclaimer  |  Comments [0]



 Monday, August 01, 2005

Found a great writeup on abstract classes versus interfaces at the code project. While not every developer or project makes use of those OOP constructs, it's important to be familiar with OOP terms for a few reasons:

  • So you can design better systems
  • So you can expand your knowledge
  • So you can better understand other developers' code
  • So you don't look like a n00b at some developer gathering

I feel that developers using Microsoft platforms are sometimes at a disadvantage when it comes to practical experience with OOP concepts. Not all developers, but if people started out coding in ASP or VB, they may not have the years of OO development that someone starting with java/etc would. I place the blame solely on the earlier MS platform and the "build it quick & dirty" ability that RAD toolkits accommodated. I'm not bashing anyone...there are many great coders on all platforms. But do you know how kludgey it was doing OO with VB6, ASP 3? Yup...pretty goofy.

Anyhow, I like simple straightforward writeups that I can print out & read on the train, etc. Hence the above link. Plus a link to CodeBetter's Feb 2005 articles -- there are some articles in there on OO concepts (abstraction, encapsulation, inheritance, and polymorphism) that will help any budding OO buddy.

August 1, 2005    Bookmark to Digg or other social bookmarking
#    Disclaimer  |  Comments [0]



 Wednesday, July 13, 2005
While doing some research for a coding standards document, I found a big wiki discussion on the pros & cons of Hungarian Notation. I started out programming with Pascal, then C (LPC actually), then C++, then VBA, then javascript, then VB, then C# & VB.NET. Maybe there were some other languages in there, too.

So, I instinctively use Hungarian Notation, yet I'm reading that it's a bad idea. That it only handles basic types, not objects (most of my variables are objects anyhow), that it obfuscates code, that it's "yucky", etc. Seems like Microsoft has gotten away from it in their coding guidelines, too.

Seems like the traditionalists are largely in the pro-HN camp, which makes me think that in 10 years you won't see very much HN at all. Although who knows what code will be like then.

Lastly, I got a kick out of the CodeSmell article. It refers to an experienced coder's ability to look at code & somehow sense that something "could" be wrong with it, without exactly knowing what. Hence the "smell" aspect ... it smells like it could be bad, but you're not as certain as if you could see it, touch it, etc.
July 13, 2005    Bookmark to Digg or other social bookmarking
#    Disclaimer  |  Comments [0]