Log in

Does that source match the binary? Part 1

Originally published at www.ikriv.com. Please leave any comments there.

A couple of weeks ago I ran into a problem: I had a DLL file, but I was not certain which version of the source code it was compiled from (yes, our release management could use some improvement).

Visual Studio debugger immediately knows if I try to give it a version of a source file that does not match the compiled binary, and shows a warning. But how does it know it?

The information about source files is stored in the PDB file (which must also exactly match the binary). PDB file format is not fully documented, but Microsoft provides at least two libraries to read it: DIA SDK, that is shipped as part of Visual Studio,and DiaSymReader interfaces that are shipped with .NET Framework. Despite being part of .NET framework, DiaSymReader interfaces are native COM interfaces, but Microsoft provides a managed wrapper as a NuGet package. 

Astonishingly, for the same PDB file these two libraries produce similar, but slightly different results. It turns out that PDB file may have two kinds of debug information, which I will call “managed” and “unmanaged”. DIA SDK accesses the “unmanaged” part. DiaSymReader accesses the “managed” part.

Here’s the summary of my findings so far:

Native C++ programs:
DIA SDK: returns list of source files with MD5 checksums.
DiaSymReader: does not work with unmanaged programs and returns an error.

Managed programs (C#, VB.NET):
DIA SDK: returns list of source files, but without checksums.
DiaSymReader: returns list of source files with SHA1 checksums (MD5 for Visual Studio 2013 and earlier).

Managed C++ programs:
DIA SDK: returns list of source files with MD5 checkums.
DiaSymReader: returns list of source files with SHA1 checksums (MD5 for Visual Studio 2013 and earlier).

Thus, managed C++ programs compiled by VS2015 contain both SHA1 and MD5 checksums of their source files in the PDB. On top of that Microsoft also has something called “Portable PDB format”, but I am not sure about the details yet.

Some other differences: DIA SDK interfaces can open a PDB file directly, or start with a binary and look for  a matching PDB in a given search path. DiaSymReader only supports the second mode: you cannot open a stand-alone PDB file without the binary.



Equal opportunity employment

Мой платеж, отправленный через Chase Online Bill Pay, где-то завис - деньги со счета сняли, а чек получателю то ли не был отправлен, то ли потерялся.

Я хотел выяснить, обналичен ли чек, и сколько времени займет получить деньги обратно, если он таки потерялся. Согласно таймеру, я что провел 23 минуты 45 секунд, общаясь сначала с роботом, а потом с дамой, которая очень хотела мне помочь, но производила впечатление аутиста-дислексика и очень плохо знала как работает их система. Внятных ответов на свои вопросы я, естественно, не получил.

Дама говорила не предложениями, а отдельными словами, как будто после каждого слова стоит точка, а также умудрилась несколько раз переставить цифры в разнообразных номерах счетов и суммах - офигительное качество для человека, которое работает в службе поддержки банка. Представление о работе банка дама имела самое туманное, и большая часть ее ответов начиналась со слова "probably".

К ее чести, надо сказать, что она была очень старательная, и несколько раз поблагодарила меня за терпение (видимо, менее выдержанные клиенты начинают на нее орать на 3-й минуте). Частично из-за ее старательности разговор и продлился 20+ минут - довольно быстро стало ясно, что это общение никуда не ведет, но мне было неудобно ее послать.

Polynesia welcomes you.
Кот грызет оставленную без присмотра мацу. В коробке оставалось три листа мацы, так от двух отгрыз сколько смог, ровненько так по краю. Боюсь, скоро откажется есть некошерные кошачьи консервы.

Originally published at www.ikriv.com. Please leave any comments there.

In my previous project we developed server-side software, that could only be deployed to production once a week. On Tuesday night you submit the binaries to Beta environment, and on Saturday they go to production, unless critical bugs are found during Beta testing. No binaries on Tuesday → no deployment on Saturday. Anything  that did not make the cut by Tuesday night would not go into the next release, end of story.

Interestingly, this lack of flexibility made our project more “agile”, because it created natural sprint boundaries. Extending the sprint by a day or two was not an option. Constantly adding new changes to the release candidate was not an option either. Deadline was king.

Some developers that just joined the project could not comprehend that this is for real, and there are virtually no exceptions. “But this is just a very small change” would not work. “But I was only an hour late” would not work either. If it was not tested and deployed to Beta by 5PM Tuesday, it is not going to production Saturday. I am not going to make people stay late, test your stuff and handle emergency tickets just because you procrastinated. Case closed.

I can’t really blame those developers; we are surrounded by ostensibly strict rules that are rarely or never enforced if the violation is minor. The vast majority of car drivers break the speed limit and roll at stop signs several times a day, but very small fraction actually gets punished. In fact, it is unsafe to follow all road rules to the letter, because people expect you to break them. Taking a pen or a box of staples from work for personal use would hardly raise any eyebrows, even though technically it is stealing. Even if discovered, it is unlikely to lead to anything bigger than a verbal warning. Stealing the same pen from a store could easily get you a criminal conviction with unpleasant long term consequences.

Going back to software development: my current project is more client oriented, with a CDN-like delivery system. So, the release schedule is much more flexible: theoretically you can whip up a new universally available version every hour or two. You may or may not call it “production”, but any client would be able to run it within minutes.

Counter-intuitively, this actually takes us farther away from agile development than weekly releases. The notion of sprint gets diluted. Everyone gives lip service to agile and Scrum, but making last minute updates to the release candidate is too easy and too tempting. Various excuses are brought forward to defend such behavior: business expects us to be flexible, we are not a software company, our software has limited distribution and extensive support, and the like. In reality, of course, this “flexibility” does not lead to faster delivery of features to the majority of clients. On the contrary, as the official final releases are delayed, most clients get their features later than expected.

Introducing non-negotiable artificial deadlines causes significant pushback. People refuse to believe that living by the book and voluntarily giving up ability to update the release candidate at any time is the good thing; that it is not only possible, but beneficial to the project. Since they have never seen it actually done, they think of it as a purely theoretical academic concept that has no place in real life. Ironically, inflexibility built into he system by “higher forces” makes it easier to eradicate procrastination and enforce true agile development.


Why banks refuse to fight fraud?

Originally published at www.ikriv.com. Please leave any comments there.

A couple of minutes ago I received a call from an automated system pretending to be from JP Morgan Chase that gave me an “online access code” and asked to call back 1-866-355-3744 in case I did not request the code.

I did not request the code, so I called back. There was another automated system that asked me for my debit card number and PIN. At this point I felt something is not kosher, hung up the phone and called Chase customer service on the back of my card.

This is when things started to get weird. The customer service representative confirmed that 1-866-355-3744 number does not belong to Chase, and calmly advised me not to give any information to anyone. She did not seem interested in pursuing the case any further. I asked to be transferred to the fraud department.

The fraud department was not much interested either. The representative asked for my debit card number and my name. I explained the situation to her, and she said she need to further verify my details to help me with any fraudulent activity on my account. I explained, that was not the purpose of my call: I did not give out any information, and all I want Chase to do is track down the people who are impersonating them, assuming that would have better connection with the FCC, or the FBI, or whoever else handles this kind of stuff.

The fraud department representative would not budge: she had the script and she had to follow it. Since the only option in the script was apparently “OMG, some people took my money!”, it felt hopeless, so I declined further assistance and hung up.

Judging from the Google search, this fraud has been going on since at least 2010, but no one seems to give a damn. I filed a complained with the FCC, let’s see if it takes us anywhere.

Rewind back a couple of months. My relative’s wallet was stolen, apparently on the bus, with a little bit of cash and a few credit cards. She reported it to the Port Authority police in New York. She had a pretty good idea who stole it and I think if they got to it, they could catch the guy. However, they were totally not interested. In fact, they outright refused to take the case and issue a police report, claiming that it is not certain that the theft occurred on the bus. A police report was still needed to trigger fraud protection from the credit card companies, so she went to her local town police. They were sympathetic, opened the case and issued the report, but indicated that they will not be able to conduct any investigation, since the theft happened outside of their jurisdiction. After the report was issued, the credit card companies promptly removed all unauthorized transactions and ate up the loss.

The bottom line is that the law enforcement and the banks for some reason seem to be completely not interested in pursuing the fraudsters and the thieves. It seems cheaper for them to just eat the loss than to do something about it.


Искусство заголовка

CNN: Fox cancels 'The O'Reilly Factor' amid harassment claims
BBC: Fox dumps O'Reilly amid harassment claims
Fox: FOX Drops O'Reilly
AFP: Bill O'Reilly will not be returning to the Fox News Channel
Lenta.ru: Оскорбивший Путина журналист Fox News уволен после обвинений в домогательствах
gazeta.ru: Хулителя Путина свалил сексуальный скандал

In uniformity we trust

Originally published at www.ikriv.com. Please leave any comments there.

Big corporations love uniformity: things are much easier to manage if they are all alike. But then they may also fail all at the same time, with catastrophic consequences. Remember the Great Famine in Ireland? The whole country was uniformly growing only potatoes, because they performed better than other crops, but then potatoes failed, and a million people died of hunger. Something similar, although thankfully not as tragic, can happen to a software company: the vendor of a widely used component may go out of business, be bought by a competitor, or change licencing terms in an unacceptable manner. If the company heavily relies on this one component for all its projects, the results may be devastating. This is especially true for closed source, paid components.

An obvious solution is to diversify: use certain component in some projects and its competitor in others (e.g. Infragistics vs DevExpress). However, this usually does not fly well with the top management, that perceives this as anarchy and waste of resources.

Another solution that I heard of was to wrap: some infrastructure group writes a layer around a heavily used component, and if this component has to be replaced, only this layer has to be re-implemented (one project), while the rest of the projects remain virtually unscathed. This solution is seriously considered and in fact is being implemented by some organizations.

However, personally I think this is not a solution at all: it’s an illusion of solution. Of course, the corporation must be really big to justify the expense of a dedicated infrastructure group, but let’s suppose it is big enough. Anyway, even with the best infrastructure group on Earth, the wrapper will

  • Lag behind the original component.
  • Have less features than the original component.
  • May still be impossible to port to a new component when the need arises.

The lag problem is obvious: it takes time to write and test the wrapper. The wrapper by necessity will abstract away some features of the original. If it blindly exposed all the features of the original, it would be impossible to port to a different component with substantially different set of features.

But even with abstraction, unless the target component is chosen ahead of time, the wrapper may still be impossible to port to a new component without loss of functionality. The new component may not expose required features at all, or its interface may have completely different philosophy behind it. Creating a unified interface that works with both the original and the new components may be as difficult or even more difficult than writing either component in the first place.

So, I am quite skeptical about a wrapper idea, except for the cases when the wrapper introduces some radically new level of abstraction and thus becomes valuable in itself.

Originally published at www.ikriv.com. Please leave any comments there.

TL;DR Dynamic “semver” versioning is evil, because it makes it too easy to break things on a global scale. With semver versioning the builds are not-repeatable: you may get one version of a module now and another version 5 seconds later. Combined with deep dependency hierarchies, this creates too many single points of failure, and periodically leads to disaster.TL;DR

Back in October 2016 I found that our software won’t compile, since a developer from Virginia just published a faulty version of a module I never heard of. But not to worry: he promised to fix it once he gets back from the gym.


Now, think about it: some guy we don’t know publishes a new version of a module we never heard of, and a couple of hours ago it breaks our compilation. If this is not scary, I don’t know what is.

There are three reasons for this debacle:

  1. The dependencies are in an external repository like npm.
  2. Versioning of the dependencies is dynamic (see the semver specification).
  3. The dependency tree is deep.

It is convenient to store dependencies in a remote repository: it greatly simplifies management of the code. However, this means that to build your code you must be connected to the remote repository, and the modules you are looking for must not be deleted. This alone has serious consequences: in 2016 npm prevented a developer from deleting his own code,  because it would cause too much trouble.

Dynamic versioning makes the problem worse: it makes the builds not repeatable. Even if you don’t touch anything in your code, what you get from the external repository may change from one moment to the next.

Deep dependency tree means that you indirectly depend on hundreds of modules. As a developer, you can tightly control your immediate dependencies by specifying their exact versions, but you don’t have a say on whether their dependencies are exact.

To get back to our story, the faulty module that was to be fixed after gym is named gulp-sourcemaps. It was a 8th order dependency, so we did not even know it existed. The chain of dependencies went something like this:

{our code} => grunt-contrib-imagemin => imagemin => imagemin-optipng => optipng-bin => bin-build => decompress => vinyl-fs => gulp-sourcemaps.

Its immediate user, vinyl-fs version 2.4.3 had the following line in their dependency list:

gulp-sourcemaps: ^1.5.2

This will pull in the latest version that is greater or equal to 1.5.2, but less than 2.0.0. On October 13, 2016 a bunch of faulty versions of gulp-sourcemaps were published, which broke vinyl-fs and everything that depended on it. The author of vinyl-fs reacted quickly (even before Mr. Nmccready got back from the gym), and made this commit: Fix: Pin gulp-sourcemaps because they are breaking things like crazy. The dependency was now pinned to version gulp-sourcemaps v1.6.0, no variation allowed.

In theory dynamic versioning is good for quick bug fixes: it is possible to fix a problem, publish a “compatible” version, and all upstream packages will automatically pick up the fix. Unfortunately, if your version is faulty, all upstream packages will also automatically pick up the fault.

In the Microsoft world, NuGet dependencies were always exact, but since Microsoft recently copy-cats everything JavaScript, NuGet dependencies now also support semver. One can argue that this is optional, and indeed it is, but if the author of your dependency chooses to use semver for his depndencies, there is nothing you can do against it: the moment it happened you lost the repeatable build.

All you can do now is hope that the guy from Virginia comes back from the gym in time before you next release.

Visual Studio 2017: are they kidding me?

Originally published at www.ikriv.com. Please leave any comments there.

I have just installed VS 2017. Resharper for VS 2017 is not available ready yet, so I decided to at least install the Productivity Power Tools. It required 3 (three) restarts of Visual Studio to complete! In VS 2015 it took one restart.

I went to the “Tools->Extensions and Updates” menu and asked to install the Productivity Power Tools. The extension manager announced that the installation will run only after I close all Visual Studio windows. Well, so I did. Then a window with a progress bar appeared, named “VSIX installer”, that then asked me again whether I want to modify Visual Studio (did’t I just request it?). I said “modify”. Then it ran some very slow installer, that enumerated each individual power tool being installed, and paused on each tool for 10-15 seconds. The installer completed and quietly disappeared (or maybe it said it was done, I don’t remember). I invoked Visual Studio again, it showed me more progress bars on startup, and then announced that the installation is completed and it wants to restart again. Is it a cool thing or what?


Is Google search getting dumber?

Originally published at www.ikriv.com. Please leave any comments there.

Is it just me, or is Google search really getting dumber? I am having more and more (perceived, not measured) difficulty finding things, and the search results become more and more stupid.

For example, if I google for Beethoven poems, hoping to find the famous Beatles joke, I get a page full of  weird unrelated stuff. The first result is titled “Beethoven’s immortal beloved letters” and does not even contain the word poem on the page! The following result is a poem about Beethoven (well, this kind of makes sense), but the next two results are from poetry search engines that proudly announce that they found zero poems by Ludwig van Beethoven (duh!). The Beatles joke is not on the first page, nor it is on the second or the third.

Search for “Beethoven poems joke” does not make it any better. And stuff like that happens more and more.


Originally published at www.ikriv.com. Please leave any comments there.

The story in a nutshell: you suddenly start receiving “405 method not allowed” response to PUT requests to your ASP.NET web service.

This problem hit us last September, but I did not blog about it then, but today we stepped on it again, so I am practically obligated to put it in the blog.

The culprit is WebDAV. If WebDAV is installed on the server, it becomes the first handler, and intercepts all PUT and DELETE requests, so they don’t even reach your ASP.NET application. The config file where this info is C:\Windows\System32\inetsrv\config\applicationHost.config.

<handlers accessPolicy="Read, Script">
<add name="WebDAV" path="*"  
       requireAccess="None" />

The simplest solution is to uninstall WebDAV. If this cannot be done, one needs to change the path="*" part.
See also: http://www.asp.net/web-api/overview/testing-and-debugging/troubleshooting-http-405-errors-after-publishing-web-api-applications.


Buggy users and soft verges

До прошлой недели я считал различия между британским и американским английским в основном стилистическими, но посетив туманный Альбион я обнаружил, что огромное количество терминов, относящихся к дорогам, автомобилям и всему, что с этим связано, решительно отличаются, вплоть до взаимной непонятности. Например, надпись в автобусе "buggy users please make space for wheelchair users" привела меня в полное замешательство.

Краткий чатлано-пацакский словарь:

BritishAmericanRussianLiteral meaning in American
buggystrollerдетская коляскаплохо работающий
car hirecar rentalпрокат автомобилей
foot pathsidewalkтротуартропинка
subwayunderpassподземный переходметро
vergeshoulderобочинапорог ("мы на пороге великого открытия")

Ах да, еще на британских дорогах как правило указывают маленькие растояния в ярдах, а на американских - в футах.


В смысле, англичане. Я сегодня затронул на работе с коллегой-индусом тему раздела Индии в 1947 году. К счастью, мы оба прогаммисты-аутисты и можем рассуждать о таких вещах абстрактно, не вцепляясь друг другу в горло.

Так вот, по его словам, в насилии и проблемах, которые произошли при разделе Индии, виноваты в первую очередь англичане. Мол, если бы не раздел, все было бы мирно. Фильм, который я посмотрел на Ютьюбе, разделяет и углубляет эту точку зрения.

Согласно фильму, англичане виноваты в том, что
- Угнетали индусов, силой навязывая свой порядок.
- Не посчитали нужным силой навести порядок, когда индусы начали резать друг друга.
- Отвели слишком мало времени на подготову раздела, что привело к хаосу.
- Отвели слишком много времени на проведение границы, что привело к насилию. Не выдержавшие напряжения индусы были практически вынуждены начать убивать друг друга.

Насколько я понял из разговоров с коллегами-индусами, это плюс-минус официальная позиция пропаганды, по крайней мере в Индии. Пакистанцы у нас тоже есть, но я с ними меньше общаюсь.

Я не к тому, что англичане были плюшевые зайки, но на полном серьезе выдвигать против них взамно исключающие обвинения - это, ИМХО, слегка перебор.

Originally published at www.ikriv.com. Please leave any comments there.

This is a short summary of differences between WCF and .NET Remoting when they are used to control a Windows service. Complete article is found here: http://www.ikriv.com/dev/dotnet/AspNetCallsWinService/index.php.

Complete code examples for WCF and Remoting are on GitHub. In the heart of these examples is the code to create client and server without config files for WCF and for Remoting. Both use named pipes, since I believe this is a more natural way to handle intra-machine communication than TCP ports.

Here are the main differences:

Interfaces: WCF requires that all interfaces be marked with [ServiceContract] attributes, and all methods be marked with [OperationContract] attribute. Remoting has no such requirement.

Data objects: In WCF data objects may be marked with [DataContract] and [DataMember] attributes, but this is optional. Remoting requires that all data objects be [Serializable]. This in turn leads to problems with ASP.NET Web API that by default does not work with [Serializable] objects as expected. This behavior must be tweaked using special configuration settings.

Security: In WCF ASP.NET-to-windows-service communication works out of the box. In Remoting it requires adding a special “authorizedGroup” attribute. However, WCF requires the client and the server to be able to share memory, which leads to weird security issues when the client and the server run under the same user, but with different UAC elevation levels.

Documentation: In both cases named pipe bindings are neglected compared to their TCP/HTTP cousins, but with Remoting being an older technology, proper documentation is harder to find.

Performance: anecdotal evidence says that WCF performs better. I suspect that it does not matter in most cases where the data is not sent back and forth in bulk.

Size of code: both WCF and Remoting can be setup purely in code, but Remoting seems to require slightly more code. For WCF all participating assemblies, including interfaces, must reference System.ServiceModel assembly. For Remoting interfaces can be designed using only standard system assemblies.

Bottom line: I lean towards recommending WCF, but watch out for those weird security issues.


C# Null Propagation

Originally published at www.ikriv.com. Please leave any comments there.

It makes things so simple, I really wish other languages had it. Today in five or six places I wrote something like


instead of

if (_innerObject != null) _innerObject.Dispose();

Long live DRY principle! Of course, our friend Juan Gandhi will probably say it is a Maybe monad (hey, how come in this day and age Chrome spell checker does not know the word “monad”?!). Anyhow, I wish this kind of shortcut were present in JavaScript. I really miss it. The proposal is already there, but I am not sure how long it will take it to get to the standard, if at all.



Crocodile stoned to death at Tunisia zoo.

Amor Ennaifer, a vet at the zoo, told AFP. "It's terrible. You cannot imagine what animals endure from some visitors," he said. "Citizens leave waste and plastic bags... They throw stones at lions and hippos." Ennaifer said the zoo has signs and guards but this was not enough, especially during school holidays.

Т.е., это были не единичные распоясавшиеся хулиганы, кидать в зоопарке в зверей камнями - это в Тунисе, похоже, систематическое явление.

Почему-то я уверен, что не будет не демонстраций PETA у тунисского посольства, не громких требований сурово наказать преступников. Во-первых тунисцы, будучи арабами, относятся к дискриминируемым народам и публично их ругать некомильфо - можно прослыть расистом и исламофобом. Во-вторых, если перестараться, они могут и в морду дать. Куда проще облить красной краской шубу какой-нибудь американской знаменитости.

Originally published at www.ikriv.com. Please leave any comments there.

I am making a little excursion into the WCF land: I need to control my Windows Service from my ASP.NET server. While experimenting, I found that new NamedPipeBinding(NetNamedPipeSecurityMode.None) on the client is not compatible with new NamedPipeBinding() on the server. If you mix them, you’ll get a ProtocolException: The requested upgrade is not supported.

Apparently, default security is “Transport”, which (theoretically) allows to encrypt and sign transmitted messages. More details in Chris Dickson’s blog. Since named pipes are local to the machine, most likely we don’t need all this security anyway.

Incredibly, whether to encrypt/sign or not to encrypt/sign a message is decided on the interface (service contract) level. I am not sure why it is so, but it smells like a leaking abstraction.

What I generally don’t like about WCF, that it feels like foreign language. It is too complex and rich with non-trivial concepts. You can study and perfect it, but if you don’t use it for couple of years, you begin to struggle and feel helpless without a dictionary.

“No usé WCF por un tiempo” or “No utilicé WCF por un tiempo”? Or is it neither?


Silent conversions are evil

Originally published at www.ikriv.com. Please leave any comments there.

SendGrid e-mail server silently converts plain text e-mails to HTML, losing line breaks in the process. They have some reasoning on why this is a good idea, but I do believe most silent conversions are evil, since they violate the rule of least astonishment.

I have added a plugin to WordPress that sends e-mail notifications to a commentator when their coment is answered (this should have been a standard feature, but that’s another matter). It almost worked: the notifications did arrive, but all squished in one line. So, if the original message was

Line One
Line Two

it would show in the e-mail as “Line One Line Two”.

Something along the way was losing the line breaks, and I had quite a few moving parts in the pipeline: the plugin, the WordPress, PHP, Apache, sendmail on my server, the mail client and any of the SMTP relays in between.

I verified that this is not the plugin, since it sends a plain text e-mail only. After some digging I noticed that the converted HTML e-mail has some embedded image that refers to “Sendgrid”. I host my blog on Azure, and Sendgrid is the mail server they offer. And indeed, it turns out Sendgrid silently converts plain text e-mails to HTML, losing line breaks in the process.

Silent conversions are evil: this is why C++ ended up having the explicit keyword. Silent conversions that lose information are especially evil. Frankly, Sendgrid should not have done it, or at least should have made it an opt-in feature, rather than the default.


State of NJ does not recognize leap years

Originally published at www.ikriv.com. Please leave any comments there.

Number of days for all residences cannot exceed 365.”. This is the error I got from NJ division of taxation web site when I tried to enter how many days I lived in each residence in 2016. I guess they just take a day off on February 29…



Веселый носорог
Yaturkenzhensirhiv - a handheld spy

Latest Month

Май 2017
Вс Пн Вт Ср Чт Пт Сб


RSS Atom
Разработано LiveJournal.com
Designed by chasethestars