The Case of the Disappearing Slack Notifications

We happened to notice that slack messages that we were used to getting were no longer showing up. This is a walk through of how they were setup, what we figured out, and what our next steps are.

Getting an app to send slack notifications

To start with you need to configure an Incoming Webhook, the essence of which involves:

• creating a Slack app
• getting the app approved
• enabling Incoming Webhooks
• then add a webhook by specifying a channel messages are sent to

Next you need to have your app generate a JSON message and post it to the url provided by the webhook. Then, violá, you have a notification in slack from your application.

So why did we stop getting some of our notifications?

We noticed that we no longer were getting notifications when some of or processes failed. We'd get success messages, so we knew some notifications were working fine, just not some of the failures.

What happens when a message is sent to a non-existant webhook?

It seems that slack has several error response. This doesn't help much if the apps aren't expecting this. So this is one thing to improve, but why would a webhook that was working start returning "no_service"?

We have had changes on the team and one notification was put together by an intern working on our team who's account was recently deactivated...

So what happens when your account is deactivated?

Slack provides some details on this, with a small teasing statement:

Apps the members set up may be disabled. You can manage and re-enable these apps if you'd like to reconnect them.

Following the link in that statement, but isn't as precise as we'd like:

...some apps they've installed (whether third-party or custom) will automatically disable. Some features remain active (bot users, slash commands, incoming webhooks, etc.), but apps that require member-specific permissions will disable entriely.

Finally, after a bit more digging some details were found in the Installing apps section of Managing Slack apps. Under the "A pleasant vacation from unexpected uninstallation" it notes how apps will not be automatically uninstalled if they limit their scope, but then it mentions a caveat:

One caveat: this exception doesn't apply to the folks who created an app or were added as an App Collaborator. When they leave, the app is still uninstalled.

So that explains why our notifications went away. Though I'm not sure how to avoid or alert on that.

Maybe collaborators?

My first thought was adding other team members to the collaborators so that more people could do the work, but the previous section implies that it might also delete it. Regardless, that's a stop gap at best.

Maybe notification on deletion?

I'm not sure who manages removing slack users, but I'm curious if it is possible to notify their former team of the apps that will be deactivated?

At least, Admins and Owners of our slack Workspace are able to reenable apps that have been deacivated.  So if we remember who set it up or to ask if anything recently got deactivated after someone has left..

Next steps

1. Improve the error handling when posting a slack message.
2. Add alerting if our apps cannot post a slack message (not just a log message).
3. Follow up with Slack Workspace Admins / Management on:
   
   1. If they can find apps owned by deactivated users
      
      • Unfortunately, they can only see the list of deactivated apps in chronological order.
   2. If we can get notified of apps that were deactivated when an account is inactivated.
      
      • The Slack Admins weren't aware of this process so more digging needs to happen to figure this out.
   3. Also worth asking these questions of Slack if I can find out where to ask them. 

Fumbling at Finding things with Selenium

Finding things with Selenium

At it's root this is easy enough.  Just call findElement() on the driver and give it some matching - we've been using css selector - and viola.  Then I wanted it to be visible - so just call .isDisplayed() on the element, but I kept getting false.

I should note that I was adding this to some functional tests being run against our hybrid mobile application.  So I could track down the css styles viewing it in a browser I'm not sure how to map the Selenium element I'm getting in our tests to the same thing.  (Any pointers would be appreciated. :))  This left me fumbling around wondering why I wasn't matching what I was seeing in the browser.

Then I thought to search for the css selector I was matching on and realized there were two of them on the page and that I need to add another part to the selector.  Not one of my proudest moments - I'd like to blame that I'm not always/often working with css and it takes a bit for my brain to get into debugging when it doesn't work off the bat.

Vagrant

So I had hoped to dig more into vagrant...and I sort of did.  I touched base with our systems team since they use puppet and I was hoping to make use of what they did to quicken things up.  Turns out they use vagrant to test out their puppet scripts, but they don't have access that they'd like to share...yet.  I'm hoping to keep poking at that and get an appropriate access created!

Grails switching to and from inline plugin usage

We had developed some local plugins that would be installed in our main app when it was deployed. However, during development it was very convient to have them inline. Unfortunately, trying to switch back to inline usage we kept seeing a message indicating that it wanted to upgrade the plugin. Saying yes would cause it to delete the local folder and download the deployed version.

We were using a settings.groovy file in our ~/.grails that indicated the location for inline usage with "grails.plugin.location.pluginname". However, since we deployed the plugin with a name like 'plugin-name' it seems the two didn't match up. By using "grails.plugin.location.'plugin-name'" instead it worked like a champ.

Hrm...wonder if some validation could be added to help out with that - though I guess most people would just be consistent with the names...*sigh*

LDAP PagedResultsResponseControl

I'm guessing there's a logic reason for it, but I find it extremely annoying that the PagedResultsResponseControl is not available until the after the results have been walked through. This forces me to handle the list in some way at the point I'm making the LDAP request (when I wouldn't think the system would be aware of the object being worked on just that it's got a query) or have some way of going back for it (which makes the rest of the system have to be aware that it's working with LDAP and paging).

Spring has a solution for this I should look at more to see if I'm missing something.

LDAP Paging

This struggle came up because I was trying to generically handle paging for either a relational database or LDAP repository, since the end user can choose either type to store their information.

Databases have several ways of handling paging, both within the query itself or, if it supports scrollable cursors, specifying the position of the cursor in the result set and getting a page of data from there.

I figured there must be something similar for LDAP repositories. Sadly, no. The closest in functionality was an RFC proposal for Virtual List Views. It wasn't accepted and while some Java API was developed for it in the JNDI booster back released for Java 1.4, I've been unable to find any updates for Java 5.

What there is something called Simple Paged Results (RFC 2696). The way it works is you specify the size of your page and the server will give you that many records and a cookie that you must pass in on future requests. You can't change the cookie, which means you're limited to only going forward. This seems a bit annoying when compared to database paging and I just figured there must be something that was equivalent.

Finally I started thinking about what I was using it for and why I would need bi-directional paging. The plan was to use this paging to get thousands to tens of thousands of records at a time. If there was a need to display small pages (like for a UI screen, then I'd let it worry about making smaller pages. The only reason I can see for needing to page backwards would be for a user at a UI screen.

How likely is it that someone is going to want (let alone need) to go through tens of thousands of entries and be able to page backwards? Something, tells me having a limit on the number of requests returned is just fine for the UI. For anything needing to get all of the data there's still the Simple Paged Results that will work just fine.

Glad I stopped fixating on the differences between RDB and LDAP paging and thinking about how it was going to be used...

Polymorphism forgetfulness

Since I just got into a silly argument and lost because I retrained myself poorly I'm going to write it down several times so I can (hopefully) remember it:

Java uses the object instance to determine which overridden method to look at and the reference of a parameter to determine which overloaded method to call.

Example:

   public class A {
       public void test(A a) { System.out.println("A from A"); }
       public void test(B b) { System.out.println("B from A"); }
   }
   public class B extends A {
       public void test(A a) { System.out.println("A from B"); }
       public void test(B b) { System.out.println("B from B"); }
   }

So which class the test method is called from depends on whether you called new A() or new B(). Which method in that class is called depends on whether you assigned the object to a reference of A or B.

Changing default locale for Java in Windows

I was trying to test something for a different locale and make the mistake of assuming that what I did actually changed it for java without confirming it. I smartened up and actually tested it and lo and behold it didn't actually change it. The correct way is to go to Control Panel -> Regional and Language Options, then update the Standards and formats to the desired language and click Apply. If you run a Java program to check out the default locale it will show the new locale.

"Controlled Versus Open" Source Control

I've started working on a project that is using ClearCase as it's source control tool and I'm still learning how to use it and what it does for us. My previous background has been mostly with CVS and dash of Subversion thrown in. My initial impression is that while CVS seems to focus on keeping it simple, ClearCase seems to be more concerned with making sure there are checks and balances.

As I was looking through the code and seeing things that, while they aren't causing any harm, probably should be fixed (like, unused imports, typos, etc...) I'm realizing that in CVS I would've just done it. By in my new world I'm going to need to submit a formal request explaining what I'm changing and why to get a approval so that someone can create a ticket that I can then check the code out against.

There are times where that checks and balance stuff is really nice - I imagine. Possibly when you're trying to change a piece of code that is central to the application. I wonder how much it squashes peoples desire to clean things up. There must be a happy middle ground, which is something I need to research and think about some more.

Blind Planning

So we finally pushed to try doing our development work in blocks of work that would be 6 weeks or less. We're also trying to push for the next block (yeah, I'd say iteration, but seems like a dirty word in my office place) to planned at the end of the first block. Unfortunately the project sponsors are looking for a project plan to be done and we have been unable to convince our project manager to put everything else in the next block as a place holder. This lead to our meeting this morning to attempt to plan out the remaining blocks of development. I suggested that if we were really going to do this that it might be better to actually have access to the requirement files so that we could double check details in the requirements instead of just a summary document (especially since the document we were attempting to plan from was not necessarily accurate with what it was showing since it is maintained separately). I got a flustered look and a restatement of the request.

Nice! My only hope is that the rest of the developers are also looking for coup and we just plan the next block after we get done the first one and let the project manager sort it out. Not the friendliest of approaches, but I feel unable to get our project manager to listen to any of our ideas.

Managers forget how it used to be...

Obviously this is not the case for everyone, but a colleague of mine who was a manager for a while before coming back to programming expressed that it is easy to forget what drives developers. Managers have their own sets of concerns that they need to drive for.

Document-itis

I'm currently on a project where it seems every time I turn around a new document has been created for a different way to view the same data. I'm not sure how to stem the tied of this swelling mountain of documents. Part of the issue is that the project is very far behind and management is desperately trying to bring in the hours yet still complete the work requested. So the thought of cleaning up the documentation and perhaps generating other views of the data gets shelved because we don't have enough time...even though we end up looking at data that is incorrect and making decisions off it because we didn't keep documents synchronized.