With the release of Jive 7 and the number of customers undergoing a Major Upgrade or onboarding a new community, we have seen an increase in requests to upgrade to minor versions such as 7.0.1 before the new site is live. To meet this demand in the most effective way possible we have developed the Fast Track Upgrade Program for Minor Upgrades.
The goal of this program is to ensure our customers have an easy and expedient option for being on the latest and most stable version available when they launch their community.
Full details on the program and how to take advantage of it can be found in the Minor Upgrade Fast Track Program Overview document. We ask that anyone who may be interested in exercising this option reads through that document in it's entirety, as the process to request a Fast Track upgrade is included there. We have also tried to address some anticipated questions in the Minor Upgrades Fast Track Program FAQ.
If you have any questions regarding this program which are not addressed in the FAQ, please submit a case in your private support group. We look forward to helping customers take full advantage of this new option.
We will be performing an upgrade of our US VPN infrastructure during the Maintenance Window on Saturday March 29th, 2014 at 10:00pm PST. It is estimated that this work will take two hours but could possibly extend up to five hours, during which all customer VPNs in the US will experience an outage.
This upgrade is necessary in order to meet network platform requirements for reliable, supportable, and secure operation of the VPN service. This upgrade also ensures accommodation for current and future VPN demands. As with most upgrades, the new version brings bug fixes, improved stability, and enhanced security.
There are no gateway changes on the customer-end required for this upgrade. Extensive testing has been performed to avoid needing customer involvement as much as possible. If any troubleshooting is required, we will contact individual customers directly.
For more details on the work being performed during this upgrade, how we will test once the upgrade is complete, and more please see the VPN Gateway Upgrades FAQ
We will be performing an upgrade of our US VPN infrastructure during the Maintenance Window on Saturday March 22nd, 2014 at 10:00pm PST. It is estimated that this work will take two hours but could possibly extend up to five hours, during which all customer VPNs in the US will experience an outage.
This upgrade is necessary in order to meet network platform requirements for reliable, supportable, and secure operation of the VPN service. This upgrade also ensures accommodation for current and future VPN demands. As with most upgrades, the new version brings bug fixes, improved stability, and enhanced security.
There are no gateway changes on the customer-end required for this upgrade. Extensive testing has been performed to avoid needing customer involvement as much as possible. If any troubleshooting is required, we will contact individual customers directly.
For more details on the work being performed during this upgrade, how we will test once the upgrade is complete, and more please see the VPN Gateway Upgrades FAQ
We will be performing an upgrade of our US VPN infrastructure during the Maintenance Window on Saturday March 8th, 2014 at 10:00pm PST. It is estimated that this work will take one hour, during which all customer VPNs in the US will experience an outage.
This upgrade is necessary in order to meet network platform requirements for reliable, supportable, and secure operation of the VPN service. This upgrade also ensures accommodation for current and future VPN demands. As with most upgrades, the new version brings bug fixes, improved stability, and enhanced security.
There are no gateway changes on the customer-end required for this upgrade. Extensive testing has been performed to avoid needing customer involvement as much as possible. If any troubleshooting is required, we will contact individual customers directly.
For more details on the work being performed during this upgrade, how we will test once the upgrade is complete, and more please see the VPN Gateway Upgrades FAQ
If you are a Hosted customer in the process of upgrading to Jive 7, please read the following message.
Beginning Friday, January 24th 2014 we will be performing a minor upgrade on 7.0 Pre-Production and UAT sites to 7.0.0.1. Groups of customer sites will be upgraded on different days, and we expect this process will extend through the week of February 3rd. This maintenance means that your upgrade sites will be unavailable for the duration of the upgrade on your particular instance, which should be less than one business day. Communications will be posted to your open upgrade cases as the maintenance commences and completes on your instance.
As we mentioned in current upgrade cases, in the interest of providing you with the best performance, security, and stability we will be installing any application updates released prior to your Test-Run.
Please let us know if you have any questions by posting to the Upgrade to 7.0: Data Copy case in your support group.
In an effort to optimize capacity and performance on our UAT infrastructure, instances that are not being used will be hibernated after a period of 14 days of inactivity. Once a system is hibernated, future access requests will be redirected to a page similar to the one pictured below with the option to reactivate the instance. If you experience any issues with a hibernated instance, please submit a support case and an engineer will be assigned to review the instance.
For additional information please see our UAT Hibernation FAQ
Jive Software is happy to announce the addition of our new Data Center space in Amsterdam!
Recently the Jive Hosting team has been coordinating and implementing services at a new data center facility located in Amsterdam and operated by Telecity Group. This is part of the continued strategic shift to improve the way Jive provides hosting services to customers as we continue to migrate to Jive-managed hosting facilities globally. Jive will own and manage all of the equipment used to support our hosted infrastructure, further enabling us to deliver a world class service.
Migrating to our new Data Center
As of now, all new EU based communities will launch from our facility in Amsterdam. Jive will also begin the process of migrating existing communities from our UK SunGard to our Amsterdam facility.
This is an exciting time for Jive, as well as it's many customers. If you have any questions please feel free to submit a case to address any issues that are not answered in our FAQ: Jive Data Center FAQ
Jive Software is excited to announce the addition of our new Data Center space at IO Data Centers in Phoenix, Arizona.
Over the past few months the Jive Hosting team has been planning and implementing services at a new data center facility located in Phoenix. This change marks a significant strategic shift in the way Jive provides hosting services to its customers. Jive will own and manage all of the equipment used to support our hosted infrastructure, further enabling us to to deliver a world class service.
Migrating to our new Data Center
Beginning in February of 2012 all new communities will launch from our facility in Phoenix. In addition to hosting all new communities in Phoenix, Jive will begin the process of migrating existing US based communities from SunGard to IO.
This is an exciting time for Jive, as well as it's many customers. I am sure you will have some questions along the way so please feel free to submit a case for any questions that are not answered in our FAQ: Jive Data Center FAQ
2011.02.22 UPDATE:
At this point all queues are clear, mail should be flowing normally without delays.
2011.08.22 UPDATE:
I wanted to update everyone on the situation. This was completely resolved over the weekend and we had no email delays. This morning at 4am we had a flood of emails destined to an external mail server that was not accepting connections, which in turn backed up our mail queues again. Emailed continued to flow through but was delayed. We have isolated the queues that are having issues and new mail is currently flowing with no delays. The existing queues should be processed in the next 1-2 hours.
2011.08.17:
Starting on 8/17, we detected an issue with one of our outbound mail cluster which caused a delay in delivery for some hosted customers. Here are the technical details:
There are multiple mail servers in this mail delivery cluster, front-ended by a load balancer. One of those servers fell into a state where it was accepting mail, but didn't seem to be attempting to deliver it - effectively just queueing the outbound email. This caused a domino affect where the mail queue for all servers became severely backed up. We disabled the node from the load balancer to give it a chance to clean up it's queue and added more capacity to handle the backlog. We still have a backlog of mail, but is quickly catching up and should be complete by this weekend.
Following the detection of the issue, we made the following changes across all the nodes:
* Tuned the mail server configuration
* Increased hardware (cpu/memory)
* Performed a full review of the current monitoring and will be adding additional monitoring to detect this type of failure
Further, we plan to move our mail cluster to physical hardware and continue to increase capacity. We sincerely apologize for any inconvenience this may have caused and rest assured that Jive takes this issue very seriously. After the above changes, we will have a much more robust and scalable outbound mail system.
Over the past few weeks, we've made some minor changes to the Jive Cloud Customer Portal along with a visual refresh of the interface. We're excited to roll the changes out and hope to hear your feedback!
You'll notice right away that we've spent some time on the visual design of the application and made some minor changes to navigation and flow.
We have additional improvements planned and you'll be seeing incremental improvements over the next few months.
Goals
Highlights
We're very excited at the updated interface and will be continuing to roll out features and improve the user experience in the upcoming months. We'd love to hear your feedback and any ideas for improvements in the future.
Hi there -
Wanted to take a minute and introduce myself to the group. As many of you know, I started with Jive on May 2 to head up the SaaS/Hosted operations group. Super excited to be here and ready to jump in - to that point, my first goals are to get an understanding of the hosting environment, continue the great work that Bob and the rest of the group has done thus far to make the environment as stable as possible and create long term plans for scalability as Jive grows.
I come to Jive from Mozilla where I ran the Engineering Operations group. I was at Mozilla for 5.5 yeas where we grew our environment from 3 racks with daisy chained 100mb switches to the 7 datacenter environment it is today where we normally push over 10gb/s including all server, network, datacenter and application architecture. I also headed up the metrics, infrasec, build/release, support, web development and services engineering groups. Previous to Mozilla I ran global datacenter operations for Macromedia, worked in a financial startup and wrote code for Cisco.
I would like to continue to solicit your input and be transparent with all of you around all things related to Jive Hosting. I will take over ownership and communication via this group to collaborate with all of you. Extremely excited to be here and help the group continue the great work they already have in progress. Please feel free to reach out to me with any questions or concerns, and look forward to working with all of you!
-Justin
Jive Customers,
Given the recent major outages, periodic node outages, and succession of weekend maintenance events, I wanted to have a central communication vehicle to communicate what we’ve done, are doing, and will continue to do in order to remediate the situation. What better place than in a social business software product…..I apologize in advance for this being a War and Peace like blog post but felt it was important to be transparent and complete. We will utilize this group going forward to share our plans with you.
Summary at a Macro Level
I am sure every one of you probably care most about 3 things:
In the back half of 2010 Jive saw unprecedented growth in the users and engagement within its existing customer instances, a shift to true mission critical usage of its software, and the addition of some of the largest enterprises and customer communities in the market. This dramatic increase in usage, storage, bandwidth, and scale exposed weaknesses within Jive’s organizational processes and hosting architecture.Follow the link to this document to understand the major areas and details we are focusing on w/r/t the above: https://community.jivesoftware.com/docs/DOC-3784
As we resolve this situation, our commitment to you is to adhere to the following principles for everything we do:
Major Outages
Recap of the 2 major outages for anyone who may not have seen them yet are here:
Outage #1 - January 14, 2011: https://community.jivesoftware.com/community/jivetalks/blog/2011/01/18/update-on-friday-s-outage-january-18-2010
Outage #2 - February 10, 2011: https://community.jivesoftware.com/community/support/blog/2011/02/11/update-on-service-interruption-on-thursday-feb-10
Both of these were due to 2 fundamental issues:
Both of these will always happen in technology environments. That the reason you have change control and redundancy built into any architecture. We have locked down our environments and our using the maintenance windows you have seen to triple check redundancy exists as it should.
Maintenance Schedules
As you have seen, we have had multiple maintenance windows over the course of the last several weekends. Each of these have been successful and well executed. To implement the needed changes, we are going to have a series of maintenance events over the course of the next several weekends as well, likely through the end of March. We will do everything possible, as fast as possible, as carefully as possible, and as non-intrusively as possible to implement the changes. Most will involve no system impact, some will require it.
Rather than just send weekly emails, I wanted to be transparent here around where we are at. I received some valid complaints from customers around lack of notice and those are valid. To take the mystery out of that, I want to get us all on the following cadence:
For those of you who have internal processes and communication lead time, you should set the above expectations now that there will be maintenance through mid-April, it will most likely not involved down time on most weekend but that there is always a potential service interruption of something unexpected occurs.
Results
It our expectation that by the middle of April the required instrumentation, architecture, capacity, redundancy, and personnel will largely be in place. We will also have tested failure scenarios to assure that we have a resilient hosting environment that is resistant to issues that will impact our customers or be visible to their end users.
We will continue to update on our progress and will be proactive in communicating our successes or any issues we may encounter.