Email Veracity Plugin 33

Posted by Carsten Nielsen
on Tuesday, November 06

Time and time again in our Rails projects we find ourselves validating email addresses for well-formedness, often in User models like this:

  validates_format_of :email, :with => /\A([^@\s]+)@((?:[-a-z0-9]+.)+[a-z]{2,})\Z/i

This is fine for checking the form of an email address, but we can do more to make sure that users enter correct addresses. Anyone who's run a production site has witnessed mistyped addresses like heycarsten@gmaol.com, and with the above scheme this would be accepted. Here's where the validates_email_veracity_of plugin comes in.

  validates_email_veracity_of :email

Now in addition to checking the format, the email domain is checked for MX records. (It's remarkable how many errors this catches on our production site Languify.)

To speed things up, the plugin includes a list of popular domains which are skipped.

Just want to check the format?

  validates_email_veracity_of :email, :domain_check => false

Require a custom error message?

  validates_email_veracity_of :email, :message => 'is not correct at all.'

By default the plugin waits for two seconds before quietly skipping the domain lookup, if you want to change this timeout interval simply:

  validates_email_veracity_of :email, :timeout => 1 # => Time in seconds.

There are many other options, all documented in the README.

Install the plugin into your Rails project the standard way:

  > script/plugin install http://svn.savvica.com/public/plugins/validates_email_veracity_of

Let us know if validates_email_veracity_of has helped you, or of any improvements that come to mind. As always, patches and enhancements are encouraged.

Comments

Leave a response

  1. Pete FordeNovember 06, 2007 @ 03:22 PM
    This is kind of awesome! I'll definitely be using this.
  2. billNovember 06, 2007 @ 05:10 PM
    Cool idea guys. FYI, the download includes and entire Rails install in /test/rails_root/vendor/rails/, which is probably unintentional.
  3. Wesley MoxamNovember 06, 2007 @ 05:50 PM
    Thanks Bill. I just fixed the issue with vendor rails, so the download is much smaller now :)
  4. Gary KingNovember 06, 2007 @ 07:53 PM
    The URL for the "script/plugin install" is the same color as the background, so that should be changed so that it's visible :)
  5. Ralph AngenendtNovember 07, 2007 @ 08:29 AM
    Looks mostly useful. But by only checking for an MX record, you are leaving out users whose domain only has an A record, which is enough for receiving mails. If a domain doesn't have an MX record, all mailers known to me are trying to reach an SMTP server at the A record of the domain. So it is completely "legal" (RFC wise) to not have MX records, even though you want to receive mail. Cheers, Ralph
  6. Carsten NielsenNovember 07, 2007 @ 09:28 AM
    Ralph, I made a design decision when making the plugin to check for MX records since it's fast and reliable but I did not know that it was RFC valid to have a domain without an MX. I am going to put it on the official TODO list to add functionality that when no MX records are present to check and see if port 25 is open and wrap it in a timeout as well. Thanks for the feedback!
  7. Ralph AngenendtNovember 07, 2007 @ 10:20 AM
    Thanks a lot. Mail sometimes isn't as simple as it looks >:)
  8. Jay LevittNovember 07, 2007 @ 09:52 PM
    Please *DON'T* go pinging the SMTP server on every validation; it's abusive and unreliable - just because the mail server is down or overloaded doesn't mean it doesn't exist. A DNS lookup is plenty - check for an MX record, and if that fails, check for an A record. Anything beyond that is diminishing returns and should be accomplished by a confirmation e-mail (which is what you really need if you want to protect against typos).
  9. Paul DowmanNovember 08, 2007 @ 04:38 PM
    Interesting, I'll definitely check this out! btw, you might find this interesting: http://haacked.com/archive/2007/08/21/i-knew-how-to-validate-an-email-address-until-i.aspx
  10. Carsten NielsenNovember 09, 2007 @ 09:53 AM
    Jay, no worries there, I have not added that functionality and when the time comes to enhance the plugin I will investigate many avenues and use the least abusive and most reliable method available. Above all, I am going to wait a while and gather feedback before I refactor anything.

    Paul, I appreciate your interest! I have tested many different patterns to include RFC 2822 validation but none of them will pass the tests. You are more than welcome to submit an enhancement.
  11. BobNovember 10, 2007 @ 05:35 AM
    You missed the largest provider of email addresses in the world in the known_domains - yahoo.com :) Seriously though, thanks for this plugin. I'll feel warmer inside when I swap my boring regexps out!
  12. Carsten NielsenNovember 11, 2007 @ 11:31 PM
    Bob, duly noted. I also left out aol.com, they are both in the list now. Thanks for the feedback!
  13. plutardNovember 13, 2007 @ 05:21 PM
    Just want to second Ralph's comment that we must check for an A record (and to not ping port 25). It's very common to not have an MX record. You should be able to just use gethostbyname(), no?
  14. ScottGNovember 14, 2007 @ 06:07 PM
    I just installed the plugin and am using it with :domain_lookup => false, but it still appears to be failing valid email addresses that don't have an MX record (specifically acts_as_authenticated's unit tests, which use example.com email addresses). I just did a recursive grep in the plugin directory for "domain_lookup" and nothing came up. Am I missing files or something?
  15. plutardNovember 14, 2007 @ 08:34 PM
    Scott, it's actually :domain_check, not :domain_lookup
  16. Carsten NielsenNovember 14, 2007 @ 08:41 PM
    ScottG, my bad, it actually is :domain_check as plutard has indicated.
  17. georgeNovember 15, 2007 @ 04:55 AM
    am using rails 1.2.5 and i got this error; Does it mean that the plugin only works with rails 1.2.3? "Cannot find gem for Rails ~>1.2.3.0: Install the missing gem with 'gem install -v=1.2.3 rails', or change environment.rb to define RAILS_GEM_VERSION with your desired version."
  18. ara.t.howardNovember 15, 2007 @ 07:01 PM
    this patch adds support to gracefully fallback to gethostbyname - i hope the formatting doesn't get horked... cfp:~ > svn diff
    
    Index: lib/validates_email_veracity_of.rb
    ===================================================================
    --- lib/validates_email_veracity_of.rb	(revision 12)
    +++ lib/validates_email_veracity_of.rb	(working copy)
    @@ -16,6 +16,7 @@
         
         require 'resolv'
         require 'timeout'
    +    require 'socket'
         
         attr_accessor :name
         
    @@ -34,6 +35,17 @@
          rescue Timeout::Error
           nil
         end
    +
    +    # Returns the array value returned by gethostbyname, if the domain does
    +    # not exist, it will return an empty array.  If it times out, nil is
    +    # returned
    +    def gethostbyname(options = {})
    +      st = Timeout::timeout(options.fetch(:timeout, 2)) do
    +        Socket.gethostbyname(name) rescue []
    +      end
    +     rescue Timeout::Error
    +      nil
    +    end
         
       end
       
    @@ -66,15 +78,16 @@
         # specified.
         def domain_has_mail_servers?(options = {})
           return true if EmailAddress.known_domains.include?(domain.name.downcase)
    -      mail_servers = domain.mail_servers(options)
    -      if mail_servers.nil?
    +      list = domain.mail_servers(options)
    +      list = domain.gethostbyname(options) if list.blank?
    +      if list.nil?
             options.fetch(:fail_on_timeout, true) ? nil : true
           else
    -        !mail_servers.empty?
    +        !list.empty?
           end
         end
         
       end
       
       
    -end
    \ No newline at end of file
    +end
    Index: lib/extensions.rb
    ===================================================================
    --- lib/extensions.rb	(revision 12)
    +++ lib/extensions.rb	(working copy)
    @@ -52,4 +52,4 @@
           
         end
       end
    -end
    \ No newline at end of file
    +end
    
    
    
    
    
  19. ShantiNovember 16, 2007 @ 02:01 PM
    Hi - seems very cool... one thing -- does it work with emails like 'foo+bar@gmail.com' ? Also do you guys use this? http://tfletcher.com/lib/rfc822.rb Seems pretty thorough.
  20. plutardNovember 16, 2007 @ 04:27 PM
    Shanti, I'd be wary of using rfc822 for validation. There are address that are deliverable, but not valid according to that rfc. For example, IIRC, something like j.c.@yahoo.com is technically invalid (the local part needs to be quoted to have it end in a period before the @), but is deliverable (at least to some domains). It's been several years since I hit that, so my memory might be faulty on the particulars there...
  21. ZubinNovember 18, 2007 @ 03:32 PM
    George, after you upgrade Rails, you need to change the line in environment.rb which defines the Rails version. It's near the top.
  22. Carsten NielsenNovember 19, 2007 @ 10:48 AM
    Ara, I've added a A-record lookup to the plugin, thanks for the patch!
  23. PetyoNovember 21, 2007 @ 12:35 PM
    For some reason, there was a problem with my setup, and the following line: require 'extensions' I did not trace what exactly it required, but it was not the intended file. Changing it to require 'email_extensions' and renaming the file fixed the issue. Thanks for your work.
  24. Carsten NielsenNovember 22, 2007 @ 10:05 AM

    Petyo, I’ve updated the plugin, as part of the update I renamed the file so hopefully this does not happen to you or anyone else anymore.

    Shanti, The plugin does validate addresses with plus signs in them. It uses the pattern from the validates_format_of example in the Rails documentation.

  25. HagsNovember 27, 2007 @ 09:01 AM

    Excellent plugin! I just downloaded and installed this baby, and it works like a champ. Well done!

  26. aiwilliamsNovember 28, 2007 @ 12:36 PM

    Nice plugin, and thanks for your work.

    If you are interested, having the knowndomains be something we can add to would be great. I have modified the plugin to support it. The patch is at http://pastie.caboo.se/123138 In my spechelper.rb, I do this:

    ValidatesEmailVeracityOf::EmailAddress.known_domains << “nomail.net”

  27. Carsten NielsenNovember 29, 2007 @ 10:56 AM

    aiwilliams, that’s a great suggestion, I’ll be adding that functionality in the next release for sure. Thanks for the support and thanks for the contribution!

  28. dirtyPeterDecember 01, 2007 @ 03:45 PM

    When I put ‘sth@’ as email I got an error [“You have a nil object when you didn’t expect it! The error occured while evaluating nil.strip”]

  29. Carsten NielsenDecember 03, 2007 @ 04:42 PM

    dirtyPeter, thanks for the find. I have fixed the issue.

  30. makuchakuJanuary 04, 2008 @ 08:06 PM

    Thanks - this plugin will solve the exact problem I am facing :)

  31. ara.t.howardJanuary 15, 2008 @ 11:38 PM
    servers.flatten!
    + servers.compact!
    

    right now you end up with servers == [nil, nil] for invalid domains

  32. Jeremy LecourJanuary 19, 2008 @ 12:18 PM

    Hi, thanks for this plugin, it really helps me out.

    You could add some french providers domains to your whitelist : %w[ free.fr wanadoo.fr orange.fr laposte.net yahoo.fr voila.fr club-internet.fr neuf.fr noos.fr tiscali.fr tele2.fr ]

  33. DougJanuary 24, 2008 @ 09:30 AM

    Found a small bug. It’s validating this email address ( “.@netinlet.com”). Great plugin - Thanks.

Comment