TITLE!

FreeBSD 10.1 + FC Target Mode + ZFS iSCSI/FC unmap/trim

by on Dec.03, 2014, under *Nix

FreeBSD 10.1 added some lovely new features, the most interesting of which to me were:

insecure_tpc
unmap

Now, using either the kernel iSCSI initiator, or FC allows for these features to be added. ZFS supporting trim on ZVol’s means, that effectively removing items from the targets would properly free the space on the underlying storage system. I’m currently running a Mellanox ConnectX-2/3 cards, direct connected for 10gbe. When using Server 2012, deleting from the iSCSI immediately calls unmap when the recycle bin is emptied.

For, fibre, simply adding the following flags:

-o insecure_tpc="on" -o unmap="on"

For kernel iSCSI:


target iqn.2012-06.com.example:unmap {
auth-group no-authentication
portal-group pg0

lun 0 {
path /dev/zvol/main/ISCSI2
size 200G
option unmap "on"
option insecure_tpc "on"
}
}

In order to get ESX 5.5 to call trim on the space you’ll need to run:

esxcli storage vmfs unmap

See here for more details.

On a side note, I also set the -o volmode=dev on the vdev’s, to keep GEOM from picking it up.

Leave a Comment more...

FreeBSD FibreChannel SAN with ZFS and VMWare with Snapshotting

by on Jun.26, 2013, under *Nix

At work I recently got to create our new SAN using 7 480GB Crucial M4 SSD’s in a RaidZ paired with 7 4TB WD RE4’s in a RaidZ2.

We initially used older QLogic 4gb 23** cards, which did not work as a target with the isp/ispfw driver. After upgrading to QLogic 2532’s we were able to get it running using the following kernel configuration options:

device isp # Qlogic family
device ispfw # Firmware for QLogic HBAs- normally a module
options ISP_TARGET_MODE

Attempting to use ZVols’ with CTLADM caused hanging issues, and we went to sparsely populated block devices.  To do so, we simply created the appropriate volumes and touched the file, eg:

touch /ssd/main/lun1

After which creating the devices was simple:

ctladm create -b block -o file=/ssd/main/lun1 -s 2199023255552

Now, here’s where things got interesting. Currently we have a database using approximately 450GB which for development purposes doesn’t change often. While trying to keep with best development practices keeping a separate build and test servers — each would need a copy of the database, and with two separate environments with two copies of the warehouse each, it totaled to over 1800GB of space required. In came ZFS Snapshotting, which I won’t explain in detail here but allows for a pool to be cloned in a read/write state. The warehouse is only read from on the other servers — allowing for a single server to have the main copy, and 3 others distributed without additional space.

Now, ESXI 5 has support for snapshotting — and when a snapshot is created it can re-import and re-signature it which allows you to manually re-import the disks to the associated machines. Unfortunately what this didn’t allow for was automatic snapshot refreshing without the snapshot presenting both the same UUID and Datastore name as the previous re-signatured store. with a bit of help from Mike La Spina we were able to find that the first 20496K of the block device on the SAN contained the signature and the datastore name. What this allowed us to do is:

  • Create the snapshot, manually import it to the ESX environment, and let ESX resignature it
  • Create a dump of the bytes using DD to backup
  • Re-snapshot the original LUN, overwrite the VMFS metadata and re-enable the LUN

The active server either needs to be shut down during this process, or the drive taken offline, VMotioned to another host in order to re-enable the drive. If the device is active, or VSphere is open on the device, it can take up to 10 minutes for ESX to recognize the store is back up. We also found that logging into a separate ESX server without an asset attached to the store, and navigating to the directory helped speed the rescan process. From there it’s as simple as:

First, create the backup of the re-signatured LUN

dd if=/ssd/server_dw/vmfs5 of=/ssd/server_dw.hex bs=20496K count=1

Then, the snapshotting process from FreeBSD (Lun 2 in this case):


ctladm remove -b block -l 2
zfs destroy ssd/server_dw
zfs destroy ssd/orig_dw@snapshot
zfs snapshot ssd/orig_dw@snapshot
zfs clone ssd/orig_dw@snapshot ssd/server_dw
dd of=/ssd/server_dw/vmfs5 if=/ssd/server_dw.hex bs=20496K count=1 conv=notrunc
ctladm create -b block -o file=/ssd/server_dw/vmfs5 -s 2199023255552 -l 2

 
It is almost instantly available on most of our hosts, but always comes back within 10 minutes after which the server can be restarted. We also disabled checkdisk on the hard drives that are using the snapshots in order to avoid long consistency checks.

Leave a Comment more...

Ubuntu FFMPEG LAME and FLAC

by on Feb.17, 2013, under *Nix

Since Ubuntu appears to have it selected as an extra, if you run into the following error:
Encoder (codec id 86017) not found
Run the following:
sudo apt-get install ffmpeg libavcodec-extra-53
Then, to convert FLAC files to mp3 within a directory:

(for FILE in *.flac ; do ffmpeg -i "$FILE" -f mp3 -aq 0 "`basename "$FILE" .flac`.mp3" || break; done)

Where aq is the audio quality (see here). Stolen from http://blogs.fsfe.org/marklindhout/2012/11/converting-flac-files-to-mp3-with-ffmpeg-and-bash/

 

Leave a Comment :, , more...

FreeBSD 9 AES-NI

by on Sep.27, 2012, under *Nix

Took me a while to figure this out, but you’ll need a custom kernel with the following:


device crypto
device cryptodev
device aesni

In your /boot/loader.conf:

aesni_load="YES"

Verify that OpenSSL has aes in cryptodev engine:

$ openssl engine -c -t
(cryptodev) BSD cryptodev engine
[RSA, DSA, DH, AES-128-CBC]
[ available ]
(dynamic) Dynamic engine loading support
[ unavailable ]

If this fails, you can reinstall openssl from /usr/src/crypto/openssl

And to test:

openssl speed -evp aes-128-cbc -elapsed

On an Ubuntu 64 bit VM running on the same host without AES-NI:

type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes
aes-128-cbc 165563.32k 169652.52k 178299.14k 180534.95k 181209.77k

And on the host:

type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes
aes-128-cbc 9604.24k 37288.16k 120273.18k 278816.94k 363101.19k

Leave a Comment more...

Assembly runtime forwarding not working without explanation

by on Mar.18, 2011, under .NET/C#, Coding, Visual Studio .NET

After hours and hours of searching between a coworker and I as to why an assembly version forwarding in this dll hell we’ve caused ourself wasn’t working in one project and it was in another, we explored every possibility we could think of:

  1. Manually searching assemblies for manual loads
  2. Searching the CLR for the location of runtime forwarding programatically (and failed)
  3. Copying and pasting the exact same text from seperate configuration files known to be working in hopes of success
  4. Sacrificing a kitten

All to no avail. After searching, and searching more I finaally ran across this awesome Microsoft Social post where someone mentioned that the configuration element having an xmlns would cause runtime to be ignored. Surely, this couldn’t be the case, but a quick test proved it to in fact be a solution.

So, to summarize:

Bad:

<configuration xmlns="http://schemas.microsoft.com/.NetConfiguration/v2.0">

Good:

<configuration>

New one to me…

Leave a Comment more...

nHibernate and Remoting/Shallow Serialization

by on Oct.28, 2009, under .NET/C#, Coding, nHibernate

So, recently had an issue with Weborb and nHibernate serializing our entire object model due to deep serialization. After much research and my lack of deep knowledge of nHibernate I came across http://trentacular.com/tag/nhibernate/. Shamelessly stolen, of course this solved my problems immediately allowing me to specify n levels of serialization to be kept down the line. I made a few alterations to allow for collections to be serialized:

                if (nhType.IsCollectionType)
                {
                    object propertyValue = propertyInfo.GetValue(unproxiedObject, null);
                    if (propertyValue == null)
                        continue;
                    if (propertyValue is PersistentBag)
                    {
                        PersistentBag holder = (PersistentBag)propertyValue;
                        for (int j = 0; j < holder.Count; j++)
                        {
                            holder[j] = holder[j].UnproxyObjectTree(sessionFactory, maxDepth - 1);
                        }

                        propertyInfo.SetValue(
                            unproxiedObject,
                            holder,
                            null);
                    }
                    else if (propertyValue is ISet)
                    {
                        ISet holder = (ISet)propertyValue;
                        ArrayList tempcoll = new ArrayList();
                        foreach (object o in (IEnumerable)propertyValue)
                        {
                            tempcoll.Add(o);
                        }
                        for (int j = 0; j < tempcoll.Count; j++)
                        {
                            tempcoll[j] = tempcoll[j].UnproxyObjectTree(sessionFactory, maxDepth - 1);
                        }
                        holder.Clear();
                        holder.AddAll(tempcoll);
                        propertyInfo.SetValue(
                            unproxiedObject,
                            holder,
                            null);
                    }
                    continue;
                }
1 Comment more...

Anonymous methods and Delegates

by on Sep.21, 2009, under .NET/C#, Coding

So, I’m currently working on a Windows forms application built by a coworker and adding some threading to it to provide a more enhanced interface.

Of course we come into the issue of thread locking, and of course Invoke must be used. We’re in .NET 3.5, might as well get some of the benefits of the latest framework, right? Well, turned out to be a bit more confusing than I first thought.

Ex:

      delegate bool GetBoolDelegate();
      int GetBoolValue()
      {
         if (!InvokeRequired)
            return (bool)vart.SelectedIndex;
         else
            return (bool)Invoke(new GetBoolDelegate(GetBoolValue), null);
      }

The pre 2.0 way of doing things, move along and try:

            bool regchecked = (bool)this.Invoke(
                  delegate()
                  { 
                     return chkRegression.Checked; 
                  }
               );

Which gives the following error:

Cannot convert anonymous method to type ‘System.Delegate’ because it is not a delegate type

There are several ways around this, the easiest is:

      public delegate bool MethodBoolInvoke();
            bool regchecked = (bool)this.Invoke(
                  (MethodBoolInvoke)delegate()
                  { 
                     return chkRegression.Checked; 
                  }
               );

And, the final simple example of invoking for general items:

this.Invoke( (MethodInvoker)delegate() {this.Cursor = Cursors.WaitCursor; });
Leave a Comment more...

Changing the Visual Studio ‘Developer Type’

by on Jun.19, 2009, under Visual Studio .NET

So, every once and a while I accidently hit the wrong setup type when opening a brand new copy of VS after my 483rd format, and F5 is no longer debug, F6 no longer rebuild, and I realize that I *should* have hit “Visual C# Developer”.

Alas, MSDN has a solution:

http://msdn.microsoft.com/en-us/library/f1z20s6z.aspx

Just keep in mind it will overwrite some other format settings if you have them set.

Leave a Comment more...

How not to lock your database server for large deletes.

by on May.14, 2009, under TSQL

So one of my coworkers recently brought down a website accidently since someone broke the session cleaning job. So, he of course runs the procedure to delete all of the old data, and kaboom it places a lock on the table while it deletes a few hundred thousand rows.

Cute except that without that table being able to be accessed no requests would technically go through. The workaround? Simply do small units of work at a time:

WHILE 1=1
BEGIN
      DELETE  TOP (1000) 
      FROM    [db].[dbo].[table]
      WHERE  Expires < @now;

      IF @@rowcount = 0 BREAK;
END
2 Comments more...

How to convert an office 2007 document to 2003.

by on Mar.30, 2009, under .NET/C#, Coding

So, backwards converting seemed to be impossible, and is give or take but if you can interact with the desktop here’s a way to do it that’s beyond ugly, cludgy, and I wish someone had a better solution… Oh, this is to interact with OFFICE12\Moc.exe, best of luck!

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Diagnostics;
using System.Runtime.InteropServices;
using System.Windows.Forms;
using System.Threading;
using System.IO;

namespace t
{
   class Program
   {

      static void Main(string[] args)
      {

         Program.ProcessChecker.Proc();
         Console.ReadLine();
      }

      static class ProcessChecker
      {

         internal static class NativeMethods
         {
            [DllImport("user32.dll")]
            public static extern bool ShowWindowAsync(IntPtr hWnd, int nCmdShow);

            [DllImport("user32.dll")]
            public static extern bool SetForegroundWindow(IntPtr hWnd);

            [DllImport("user32.dll")]
            public static extern bool EnumWindows(EnumWindowsProcDel lpEnumFunc,
                Int32 lParam);

            [DllImport("user32.dll")]
            public static extern uint GetWindowThreadProcessId(IntPtr hWnd,
                ref Int32 lpdwProcessId);

            [DllImport("user32.dll")]
            public static extern int GetWindowText(IntPtr hWnd, StringBuilder lpString,
                Int32 nMaxCount);

            [DllImport("user32.dll")]
            public static extern bool SetWindowText(IntPtr hWnd, string lpString);

            [DllImport("user32.dll")]
            public static extern int GetFocus();

            [DllImport("user32.dll")]
            public static extern bool AttachThreadInput(uint idAttach, uint idAttachTo, bool fAttach);

            [DllImport("kernel32.dll")]
            public static extern uint GetCurrentThreadId();

            [DllImport("user32.dll")]
            public static extern uint GetWindowThreadProcessId(int hWnd, int ProcessId);

            [DllImport("user32.dll")]
            public static extern int GetForegroundWindow();

            [DllImport("user32.dll", CharSet = CharSet.Auto, SetLastError = false)]
            public static extern int SendMessage(int hWnd, int Msg, int wParam, StringBuilder lParam);

            [DllImport("user32.dll", CharSet = CharSet.Auto)]
            public static extern int GetClassName(IntPtr hWnd, StringBuilder lpClassName, int nMaxCount);

            [DllImport("user32.dll", SetLastError = true)]
            public static extern IntPtr GetWindow(IntPtr hWnd, uint uCmd);

            public enum GetWindow_Cmd : uint
            {
               GW_HWNDFIRST = 0,
               GW_HWNDLAST = 1,
               GW_HWNDNEXT = 2,
               GW_HWNDPREV = 3,
               GW_OWNER = 4,
               GW_CHILD = 5,
               GW_ENABLEDPOPUP = 6
            }

            public const int SW_SHOWNORMAL = 1;
         }
         const int WM_SETTEXT = 12;
         const int WM_GETTEXT = 13;
         public delegate bool EnumWindowsProcDel(IntPtr hWnd, Int32 lParam);

         static private bool EnumWindowsProc(IntPtr hWnd, Int32 lParam)
         {
            int processId = 0;
            uint remoteThreadId = NativeMethods.GetWindowThreadProcessId(hWnd, ref processId);
            StringBuilder WindowText = new StringBuilder(5000);

            NativeMethods.GetWindowText(hWnd, WindowText, 5000);
            StringBuilder v_szbTest2 = new StringBuilder(99);
            NativeMethods.GetClassName(hWnd, v_szbTest2, 99);
            if (WindowText.ToString().Equals("Save As") && processId.Equals(lParam))
            {
               uint currentThreadId = NativeMethods.GetCurrentThreadId();
               NativeMethods.AttachThreadInput(remoteThreadId, currentThreadId, true);

               IntPtr i = IntPtr.Zero;
               i = NativeMethods.GetWindow(hWnd, (uint)NativeMethods.GetWindow_Cmd.GW_CHILD);
               for (; i.ToInt32() > 0; )
               {

                  i = NativeMethods.GetWindow(i, (uint)NativeMethods.GetWindow_Cmd.GW_HWNDNEXT);
                  StringBuilder v_szbTest3 = new StringBuilder(99);
                  NativeMethods.GetClassName(i, v_szbTest3, 99);

                  StringBuilder builder = new StringBuilder(500);
                  builder.Append("c:\\Hey look a text.xls");
                  if (v_szbTest3.ToString().Equals("ComboBoxEx32"))
                  {
                     NativeMethods.SendMessage(i.ToInt32(), WM_SETTEXT, builder.Capacity, builder);
                  }
                  StringBuilder v_szbText = new StringBuilder(256);
                  NativeMethods.GetWindowText(i, v_szbText, 256);
                  if (v_szbTest3.ToString().Equals("Button") && v_szbText.ToString().Equals("&Save"))
                  {
                     //Save!
                     var BM_CLICK = 0xf5;
                     NativeMethods.SendMessage(i.ToInt32(), BM_CLICK, 0, builder);
                     NativeMethods.AttachThreadInput(remoteThreadId, currentThreadId, false);
                     //Lets wait till we can access the file.

                     while (true)
                     {
                        try
                        {
                           FileStream fso = File.Open(builder.ToString() + ".xls", FileMode.Open, FileAccess.Write);
                           fso.Close();
                           //kill our proc
                           Process moc = Process.GetProcessById(lParam);
                           moc.Kill();
                           return true;
                        }
                        catch (IOException x)
                        {
                           Console.WriteLine(x);
                           Thread.Sleep(10);
                        }
                     }

                  }

               }

            }
            return true;
         }

         static public bool Proc()
         {

            Process[] v_oProcs = Process.GetProcesses();
            foreach (Process proc in v_oProcs)
            {
               if (proc.ProcessName.Equals("Moc"))
               {
                  NativeMethods.EnumWindows(new EnumWindowsProcDel(EnumWindowsProc),
                      proc.Id);
                  return false;
               }
            }
            return true;
         }
      }

      #region EnumWindows
      /// 
      /// EnumWindows wrapper for .NET
      /// 
      public class EnumWindows
      {
         #region Delegates
         private delegate int EnumWindowsProc(IntPtr hwnd, int lParam);
         #endregion

         #region UnManagedMethods
         private class UnManagedMethods
         {
            [DllImport("user32")]
            public extern static int EnumWindows(
                EnumWindowsProc lpEnumFunc,
                int lParam);
            [DllImport("user32")]
            public extern static int EnumChildWindows(
                IntPtr hWndParent,
                EnumWindowsProc lpEnumFunc,
                int lParam);
         }
         #endregion

         #region Member Variables
         private List<IntPtr> items = null;
         #endregion

         /// 
         /// Returns the collection of windows returned by
         /// GetWindows
         /// 
         public List<IntPtr> Items
         {
            get
            {
               return this.items;
            }
         }

         /// 
         /// Gets all top level windows on the system.
         /// 
         public void GetWindows()
         {
            this.items = new List<IntPtr>();
            UnManagedMethods.EnumWindows(
                new EnumWindowsProc(this.WindowEnum),
                0);
         }

         /// 
         /// Gets all child windows of the specified window
         /// 
         /// Window Handle to get children for
         public void GetWindows(
             IntPtr hWndParent)
         {
            this.items = new List<IntPtr>();
            UnManagedMethods.EnumChildWindows(
                hWndParent,
                new EnumWindowsProc(this.WindowEnum),
                0);
         }

         #region EnumWindows callback
         /// 
         /// The enum Windows callback.
         /// 
         /// Window Handle
         /// Application defined value
         /// 1 to continue enumeration, 0 to stop
         private int WindowEnum(
             IntPtr hWnd,
             int lParam)
         {
            if (this.OnWindowEnum(hWnd))
            {
               return 1;
            }
            else
            {
               return 0;
            }
         }
         #endregion

         /// 
         /// Called whenever a new window is about to be added
         /// by the Window enumeration called from GetWindows.
         /// If overriding this function, return true to continue
         /// enumeration or false to stop.  If you do not call
         /// the base implementation the Items collection will
         /// be empty.
         /// 
         /// Window handle to add
         /// True to continue enumeration, False to stop
         protected virtual bool OnWindowEnum(
             IntPtr hWnd)
         {
            items.Add(hWnd);
            return true;
         }

         #region Constructor, Dispose
         public EnumWindows()
         {
            // nothing to do
         }
         #endregion
      }
      #endregion EnumWindows


   }

}
Leave a Comment :, , more...

Looking for something?

Use the form below to search the site:

Still not finding what you're looking for? Drop a comment on a post or contact us so we can take care of it!

Visit our friends!

A few highly recommended friends...