Skip navigation

Jive Hosting Group

12 posts

With the release of Jive 7 and the number of customers undergoing a Major Upgrade or onboarding a new community, we have seen an increase in requests to upgrade to minor versions such as 7.0.1 before the new site is live. To meet this demand in the most effective way possible we have developed the Fast Track Upgrade Program for Minor Upgrades.

 

The goal of this program is to ensure our customers have an easy and expedient option for being on the latest and most stable version available when they launch their community.

 

Full details on the program and how to take advantage of it can be found in the Minor Upgrade Fast Track Program Overview document. We ask that anyone who may be interested in exercising this option reads through that document in it's entirety, as the process to request a Fast Track upgrade is included there. We have also tried to address some anticipated questions in the Minor Upgrades Fast Track Program FAQ.

 

If you have any questions regarding this program which are not addressed in the FAQ, please submit a case in your private support group. We look forward to helping customers take full advantage of this new option. 

We will be performing an upgrade of our US VPN infrastructure during the Maintenance Window on Saturday March 29th, 2014 at 10:00pm PST. It is estimated that this work will take two hours but could possibly extend up to five hours, during which all customer VPNs in the US will experience an outage.

 

This upgrade is necessary in order to meet network platform requirements for reliable, supportable, and secure operation of the VPN service. This upgrade also ensures accommodation for current and future VPN demands.  As with most upgrades, the new version brings bug fixes, improved stability, and enhanced security.


There are no gateway changes on the customer-end required for this upgrade. Extensive testing has been performed to avoid needing customer involvement as much as possible. If any troubleshooting is required, we will contact individual customers directly.

 

For more details on the work being performed during this upgrade, how we will test once the upgrade is complete, and more please see the VPN Gateway Upgrades FAQ

We will be performing an upgrade of our US VPN infrastructure during the Maintenance Window on Saturday March 22nd, 2014 at 10:00pm PST. It is estimated that this work will take two hours but could possibly extend up to five hours, during which all customer VPNs in the US will experience an outage.

 

This upgrade is necessary in order to meet network platform requirements for reliable, supportable, and secure operation of the VPN service. This upgrade also ensures accommodation for current and future VPN demands.  As with most upgrades, the new version brings bug fixes, improved stability, and enhanced security.


There are no gateway changes on the customer-end required for this upgrade. Extensive testing has been performed to avoid needing customer involvement as much as possible. If any troubleshooting is required, we will contact individual customers directly.

 

For more details on the work being performed during this upgrade, how we will test once the upgrade is complete, and more please see the VPN Gateway Upgrades FAQ

VPN Gateway Upgrades

Posted by nmiller0113 Feb 21, 2014

We will be performing an upgrade of our US VPN infrastructure during the Maintenance Window on Saturday March 8th, 2014 at 10:00pm PST. It is estimated that this work will take one hour, during which all customer VPNs in the US will experience an outage.

 

This upgrade is necessary in order to meet network platform requirements for reliable, supportable, and secure operation of the VPN service. This upgrade also ensures accommodation for current and future VPN demands.  As with most upgrades, the new version brings bug fixes, improved stability, and enhanced security.


There are no gateway changes on the customer-end required for this upgrade. Extensive testing has been performed to avoid needing customer involvement as much as possible. If any troubleshooting is required, we will contact individual customers directly.

 

For more details on the work being performed during this upgrade, how we will test once the upgrade is complete, and more please see the VPN Gateway Upgrades FAQ

If you are a Hosted customer in the process of upgrading to Jive 7, please read the following message.

 

Beginning Friday, January 24th 2014 we will be performing a minor upgrade on 7.0 Pre-Production and UAT sites to 7.0.0.1. Groups of customer sites will be upgraded on different days, and we expect this process will extend through the week of February 3rd. This maintenance means that your upgrade sites will be unavailable for the duration of the upgrade on your particular instance, which should be less than one business day. Communications will be posted to your open upgrade cases as the maintenance commences and completes on your instance.

 

As we mentioned in current upgrade cases, in the interest of providing you with the best performance, security, and stability we will be installing any application updates released prior to your Test-Run.

 

Please let us know if you have any questions by posting to the Upgrade to 7.0: Data Copy case in your support group.


In an effort to optimize capacity and performance on our UAT infrastructure, instances that are not being used will be hibernated after a period of 14 days of inactivity. Once a system is hibernated, future access requests will be redirected to a page similar to the one pictured below with the option to reactivate the instance. If you experience any issues with a hibernated instance, please submit a support case and an engineer will be assigned to review the instance.

Screen Shot 2013-08-21 at 9.07.19 AM.png

For additional information please see our UAT Hibernation FAQ

Jive Software is happy to announce the addition of our new Data Center space in Amsterdam!

 

Recently the Jive Hosting team has been coordinating and implementing services at a new data center facility located in Amsterdam and operated by Telecity Group. This is part of the continued strategic shift to improve the way Jive provides hosting services to customers as we continue to migrate to Jive-managed hosting facilities globally. Jive will own and manage all of the equipment used to support our hosted infrastructure, further enabling us to deliver a world class service.

 

Migrating to our new Data Center

As of now, all new EU based communities will launch from our facility in Amsterdam. Jive will also begin the process of migrating existing communities from our UK SunGard to our Amsterdam facility.

  • The migration for existing communities will take place during Q1 of 2013 and is targeted for completion by the end of the quarter.
  • More detailed communications regarding community migrations will be delivered in the coming weeks.

 

This is an exciting time for Jive, as well as it's many customers. If you have any questions please feel free to submit a case to address any issues that are not answered in our FAQ: Jive Data Center FAQ

Jive Software is excited to announce the addition of our new Data Center space at IO Data Centers in Phoenix, Arizona.

 

Over the past few months the Jive Hosting team has been planning and implementing services at a new data center facility located in Phoenix. This change marks a significant strategic shift in the way Jive provides hosting services to its customers. Jive will own and manage all of the equipment used to support our hosted infrastructure, further enabling us to to deliver a world class service.

 

Migrating to our new Data Center

Beginning in February of 2012 all new communities will launch from our facility in Phoenix.  In addition to hosting all new communities in Phoenix, Jive will begin the process of migrating existing US based communities from SunGard to IO.

  • The migration window for existing communities will begin in Q2 of 2012 and extend through Q1 of 2013
  • More detailed communications regarding community migrations will be delivered in the coming weeks and months

 

This is an exciting time for Jive, as well as it's many customers. I am sure you will have some questions along the way so please feel free to submit a case for any questions that are not answered in our FAQ: Jive Data Center FAQ

2011.02.22 UPDATE:

 

At this point all queues are clear, mail should be flowing normally without delays. 

 

2011.08.22 UPDATE:

 

I wanted to update everyone on the situation.  This was completely resolved over the weekend and we had no email delays.  This morning at 4am we had a flood of emails destined to an external mail server that was not accepting connections, which in turn backed up our mail queues again.  Emailed continued to flow through but was delayed.  We have isolated the queues that are having issues and new mail is currently flowing with no delays.  The existing queues should be processed in the next 1-2 hours.

 

2011.08.17:

 

Starting on 8/17, we detected an issue with one of our  outbound mail cluster which caused a delay in delivery for some hosted customers.  Here are the technical details:

 

There are multiple mail servers in this mail delivery cluster, front-ended by a load balancer.  One of those  servers fell into a state where it was accepting mail, but didn't seem to be attempting to deliver it - effectively just queueing the outbound email.  This caused a domino affect where the mail queue for all servers became severely backed up.  We disabled the node from the load balancer to give it a chance to clean up it's queue and added more capacity to handle the backlog.  We still have a backlog of mail, but is quickly catching up and should be complete by this weekend.

 

Following the detection of the issue, we made the following  changes across all the nodes:

* Tuned the mail server configuration

* Increased hardware (cpu/memory)

* Performed a full review of the current monitoring and will be adding additional monitoring to detect this type of failure

 

Further, we plan to move our mail cluster to physical hardware and continue to increase capacity.  We sincerely apologize for any inconvenience this may have caused and rest assured that Jive takes this issue very seriously.  After the above changes, we will have a much more robust and scalable outbound mail system.

Over the past few weeks, we've made some minor changes to the Jive Cloud Customer Portal along with a visual refresh of the interface.  We're excited to roll the changes out and hope to hear your feedback!

You'll notice right away that we've spent some time on the visual design of the application and made some minor changes to navigation and flow.

We have additional improvements planned and you'll be seeing incremental improvements over the next few months.

 

Goals

  • Improve messaging and feedback to users throughout the application
  • Consolidate menus and provide for a more pleasant user experience
  • Improved help text and global link to help text

 

Highlights

  • Refreshed visual design and user experience
  • Improved messaging including: visual indication that you are working in a production installation, consistent user feedback on all actions, release notes and system messages pinned to the landing page
  • Ability to enable/disable maintenance mode on an installation
  • Global navigation to help text

 

Landing Page

Screen shot 2011-07-11 at 6.01.05 PM.png

  • The updated homepage sports a new global header and navigation that users can quickly access major actions within the customer portal.
  • Notifications are shown in the right column.
  • Release notes feature will display release notes for a new release when a user logs in for the first time after the release.
  • Maintenance mode state is shown on the homepage, along with a link to preview the site.

Header

Screen shot 2011-07-11 at 6.02.04 PM.png

  • Customer name and drop down select menu listing all the installations the user has access to modify.
  • Color coding to highlight whether the user is in a production instance.
  • Global link to help text, settings still includes only Password changes, but we're preparing for additional profile/access information.

Recent Activity

Screen shot 2011-07-11 at 6.03.00 PM.png

  • Recent activity now indicates is an artifact was associated with the activity
  • Elapsed time is included to show the duration of the activity

Update your site

Screen shot 2011-07-11 at 6.06.45 PM.png

  • Files are now queued using a list build element and can be removed prior to the update

Maintenance Mode

Screen shot 2011-07-11 at 6.09.28 PM.png

  • ability to enable or disable maintenance mode from this screen, we are capturing the current state of the site and will display this on the homepage.
  • we have combined maintenance mode and uploading a custom maintenance .zip file into one screen

 

Summary

We're very excited at the updated interface and will be continuing to roll out features and improve the user experience in the upcoming months. We'd love to hear your feedback and any ideas for improvements in the future.

Hi there -

Wanted to take a minute and introduce myself to the group.  As many of you know, I started with Jive on May 2 to head up the SaaS/Hosted operations group.  Super excited to be here and ready to jump in - to that point, my first goals are to get an understanding of the hosting environment, continue the great work that Bob and the rest of the group has done thus far to make the environment as stable as possible and create long term plans for scalability as Jive grows.

I come to Jive from Mozilla where I ran the Engineering Operations group.  I was at Mozilla for 5.5 yeas where we grew our environment from 3 racks with daisy chained 100mb switches to the 7 datacenter environment it is today where we normally push over 10gb/s including all server, network, datacenter and application architecture.  I also headed up the metrics, infrasec, build/release, support, web development and services engineering groups.  Previous to Mozilla I ran global datacenter operations for Macromedia, worked in a financial startup and wrote code for Cisco.

 

I would like to continue to solicit your input and be transparent with all of you around all things related to Jive Hosting. I will take over ownership and communication via this group to collaborate with all of you.  Extremely excited to be here and help the group continue the great work they already have in progress.  Please feel free to reach out to me with any questions or concerns, and look forward to working with all of you!

-Justin

Jive Customers,

 

Given the recent major outages, periodic node outages, and succession of weekend maintenance events, I wanted to have a central communication vehicle to communicate what we’ve done, are doing, and will continue to do in order  to remediate the situation.  What better place than in a social  business software product…..I apologize in advance for this being a War  and Peace like blog post but felt it was important to be transparent and complete.  We will utilize this group going forward to share our plans with you.

 

Summary at a Macro Level

 

I am sure every one of you probably care most about 3 things:

  1. How did we get here?
  2. What is the current situation?
  3. How is Jive going to make it better?

 

In the back half of 2010 Jive saw unprecedented growth in the users and  engagement within its existing customer instances, a shift to true  mission critical usage of its software, and the addition of some of the  largest enterprises and customer communities in the market.  This  dramatic increase in usage, storage, bandwidth, and scale exposed  weaknesses within Jive’s organizational processes and hosting  architecture.Follow the link to this document to understand the major areas and details we are focusing on w/r/t the above: https://community.jivesoftware.com/docs/DOC-3784

 

As we resolve this situation, our commitment to you is to adhere to the following principles for everything we do:

  • Be Transparent
  • Be Responsive
  • Produce Results

 

Major Outages

 

Recap of the 2 major outages for anyone who may not have seen them yet are here:

 

Outage #1 - January 14, 2011: https://community.jivesoftware.com/community/jivetalks/blog/2011/01/18/update-on-friday-s-outage-january-18-2010

 

Outage #2 - February 10, 2011: https://community.jivesoftware.com/community/support/blog/2011/02/11/update-on-service-interruption-on-thursday-feb-10

 

Both of these were due to 2 fundamental issues:

  1. Human error
  2. Technology failure

 

Both of these will always happen in technology environments.  That the reason you have change control and redundancy built into any architecture.  We have locked down our environments and our using the maintenance windows you have seen to triple check redundancy exists as it should.

 

Maintenance Schedules

 

As you have seen, we have had multiple maintenance windows over the course of the last several weekends.   Each of these have been successful and well executed.  To implement the needed changes, we are going to have a series of maintenance events over the course of the next several weekends as well, likely through the end of March.  We will do everything possible, as fast as possible, as carefully as possible, and as non-intrusively as possible to implement the changes.  Most will involve no system impact, some will require it.

 

Rather than just send weekly emails, I wanted to be transparent here around where we are at.   I received some valid complaints from customers around lack of notice and those are valid.  To take the mystery out of that, I want to get us all on the following cadence:

 

  • Let you see our current master maintenance schedule which is located here: https://community.jivesoftware.com/docs/DOC-37847
  • Every Wednesday we will publish 
    • Final scope of what we are planning for the subsequent weekend
    • This should be what is in the master schedule save any tweaking we have decided to do and not change materially
    • What the potential impact will be and the hours impact could occur (example, a failover test should not affect you, unless it for some reason fails)
  • A detailed schedule published Thursday evening detailing all of the timing and detailed steps
  • The ability to follow progress real time if you should so choose to as we progress through the maintenance activities
  • Every Monday we will publish a full debrief on all of the previous weekends maintenance activities and then follow the communication cadence again for the following week

 

For those of you who have internal processes and communication lead time, you should set the above expectations now that there will be maintenance through mid-April, it will most likely not involved down time on most weekend but that there is always a potential service interruption of something unexpected occurs.

 

Results

 

It our expectation that by the middle of April the required instrumentation, architecture, capacity, redundancy, and personnel will largely be in place. We will also have tested failure scenarios to assure that we have a resilient hosting environment that is resistant to issues that will impact our customers or be visible to their end users.

 

We will continue to update on our progress and will be proactive in communicating our successes or any issues we may encounter.