Merging performance branch into master (#4268)

* improve performance on sendNotifications

* Fix SQL queries

* Remove get direct profiles, not needed anymore

* Add raw data to error details if AppError fails to decode

* men

* Fix decode (#4052)

* Fixing json decode

* Adding unit test

* Initial work for client scaling (#4051)

* Begin adding paging to profiles API

* Added more paging functionality

* Finish hooking up admin console user lists

* Add API for searching users and add searching to all user lists

* Add lazy loading of profiles

* Revert config.json

* Fix unit tests and some style issues

* Add GetProfilesFromList to Go driver and fix web unit test

* Update etag for GetProfiles

* Updating ui for filters and pagination (#4044)

* Updating UI for pagination

* Adjusting margins for filter row

* Adjusting margin for specific modals

* Adding relative padding to system console

* Adjusting responsive view

* Update client user tests

* Minor fixes for direct messages modal (#4056)

* Remove some unneeded initial load calls (#4057)

* UX updates to user lists, added smart counts and bug fixes (#4059)

* Improved getExplicitMentions and unit tests (#4064)

* Refactor getting posts to lazy load profiles correctly (#4062)

* Comment out SetActiveChannel test (#4066)

* Profiler cpu, block, and memory profiler. (#4081)

* Fix TestSetActiveChannel unit test (#4071)

* Fixing build failure caused by dependancies updating (#4076)

* Adding profiler

* Fix admin_team_member_dropdown eslint errors

* Bumping session cache size (#4077)

* Bumping session cache size

* Bumping status cache

* Refactor how the client handles channel members to be large team friendly (#4106)

* Refactor how the client handles channel members to be large team friendly

* Change Id to ChannelId in ChannelStats model

* Updated getChannelMember and getProfilesByIds routes to match proposal

* Performance improvements (#4100)

* Performance improvements

* Fixing re-connect issue

* Fixing error message

* Some other minor perf tweaks

* Some other minor perf tweaks

* Fixing config file

* Fixing buffer size

* Fixing web socket send message

* adding some error logging

* fix getMe to be user required

* Fix websocket event for new user

* Fixing shutting down

* Reverting web socket changes

* Fixing logging lvl

* Adding caching to GetMember

* Adding some logging

* Fixing caching

* Fixing caching invalidate

* Fixing direct message caching

* Fixing caching

* Fixing caching

* Remove GetDirectProfiles from initial load

* Adding logging and fixing websocket client

* Adding back caching from bad merge.

* Explicitly close go driver requests (#4162)

* Refactored how the client handles team members to be more large team friendly (#4159)

* Refactor getProfilesForDirectMessageList API into getAllProfiles API

* Refactored how the client handles team members to be more large team friendly

* Fix js error when receiving a notification

* Fix JS error caused by current user being overwritten with sanitized version (#4165)

* Adding error message to status failure (#4167)

* Fix a few bugs caused by client scaling refactoring (#4170)

* When there is no read replica, don't open a second set of connections to the master database (#4173)

* Adding connection tacking to stats (#4174)

* Reduce DB writes for statuses and other status related changes (#4175)

* Fix bug preventing opening of DM channels from more modal (#4181)

* 	Fixing socket timing error (#4183)

* Fixing ping/pong handler

* Fixing socket timing error

* Commenting out status broadcasting

* Removing user status changes

* Removing user status changes

* Removing user status changes

* Removing user status changes

* Adding DoPreComputeJson()

* Performance improvements (#4194)

* * Fix System Console Analytics queries
* Add db.SetConnMaxLifetime to 15 minutes
* Add "net/http/pprof" for profiling
* Add FreeOSMemory() to manually release memory on reload config

* Add flag to enable http profiler

* Fix memory leak (#4197)

* Fix memory leak

* removed unneeded nil assignment

* Fixing go routine leak (#4208)

* Merge fixes

* Merge fix

* Refactored statuses to be queried by the client rather than broadcast by the server (#4212)

* Refactored server code to reduce status broadcasts and to allow getting statuses by IDs

* Refactor client code to periodically fetch statuses

* Add store unit test for getting statuses by ids

* Fix status unit test

* Add getStatusesByIds REST API and move the client over to use that instead of the WebSocket

* Adding multiple threads to websocket hub (#4230)

* Adding multiple threads to websocket hub

* Fixing unit tests

* Fixing so websocket connections from the same user end up in the same… (#4240)

* Fixing so websocket connections from the same user end up in the same list

* Removing old comment

* Refactor user autocomplete to query the server (#4239)

* Add API for autocompleting users

* Converted at mention autocomplete to query server

* Converted user search autocomplete to query server

* Switch autocomplete API naming to use term instead of username

* Split autocomplete API into two, one for channels and for teams

* Fix copy/paste error

* Some final client scaling fixes (#4246)

* Add lazy loading of profiles to integration pages

* Add lazy loading of profiles to emoji page

* Fix JS error when receiving post in select team menu and also clean up channel store

4 files changed, 118 insertions, 102 deletions

diff --git a/webapp/components/suggestion/at_mention_provider.jsx b/webapp/components/suggestion/at_mention_provider.jsx
index 9998e6357..d4f441f98 100644
--- a/webapp/components/suggestion/at_mention_provider.jsx
+++ b/webapp/components/suggestion/at_mention_provider.jsx
@@ -1,19 +1,19 @@
 // Copyright (c) 2015 Mattermost, Inc. All Rights Reserved.
 // See License.txt for license information.
 
-import React from 'react';
+import Suggestion from './suggestion.jsx';
 
-import SuggestionStore from 'stores/suggestion_store.jsx';
 import ChannelStore from 'stores/channel_store.jsx';
-import UserStore from 'stores/user_store.jsx';
+
+import {autocompleteUsersInChannel} from 'actions/user_actions.jsx';
+
+import AppDispatcher from 'dispatcher/app_dispatcher.jsx';
 import * as Utils from 'utils/utils.jsx';
 import Client from 'client/web_client.jsx';
-import Constants from 'utils/constants.jsx';
+import {Constants, ActionTypes} from 'utils/constants.jsx';
 
+import React from 'react';
 import {FormattedMessage} from 'react-intl';
-import Suggestion from './suggestion.jsx';
-
-const MaxUserSuggestions = 40;
 
 class AtMentionSuggestion extends Suggestion {
     render() {
@@ -99,92 +99,66 @@ class AtMentionSuggestion extends Suggestion {
     }
 }
 
-function filterUsersByPrefix(users, prefix, limit, type) {
-    const filtered = [];
-
-    for (const id of Object.keys(users)) {
-        if (filtered.length >= limit) {
-            break;
-        }
-
-        const user = users[id];
-
-        if (user.delete_at > 0) {
-            continue;
-        }
-
-        if (user.username.startsWith(prefix) ||
-            (user.first_name && user.first_name.toLowerCase().startsWith(prefix)) ||
-            (user.last_name && user.last_name.toLowerCase().startsWith(prefix)) ||
-            (user.nickname && user.nickname.toLowerCase().startsWith(prefix))) {
-            // create a new object here since we're mutating it by adding the type field
-            filtered.push(Object.assign({}, user, {type}));
-        }
-    }
-
-    return filtered;
-}
-
 export default class AtMentionProvider {
     constructor(channelId) {
         this.channelId = channelId;
+        this.timeoutId = '';
+    }
+
+    componentWillUnmount() {
+        clearTimeout(this.timeoutId);
     }
 
     handlePretextChanged(suggestionId, pretext) {
+        clearTimeout(this.timeoutId);
+
         const captured = (/@([a-z0-9\-\._]*)$/i).exec(pretext.toLowerCase());
         if (captured) {
             const prefix = captured[1];
 
-            // Group users into members and nonmembers of the channel.
-            const users = UserStore.getActiveOnlyProfiles(true);
-            const channelMembers = {};
-            const channelNonmembers = users;
-            if (this.channelId != null) {
-                const extraInfo = ChannelStore.getExtraInfo(this.channelId);
-                for (let i = 0; i < extraInfo.members.length; i++) {
-                    const id = extraInfo.members[i].id;
-                    if (users[id]) {
-                        channelMembers[id] = users[id];
-                        Reflect.deleteProperty(channelNonmembers, id);
+            function autocomplete() {
+                autocompleteUsersInChannel(
+                    prefix,
+                    this.channelId,
+                    (data) => {
+                        const members = data.in_channel;
+                        for (const id of Object.keys(members)) {
+                            members[id].type = Constants.MENTION_MEMBERS;
+                        }
+
+                        const nonmembers = data.out_of_channel;
+                        for (const id of Object.keys(nonmembers)) {
+                            nonmembers[id].type = Constants.MENTION_NONMEMBERS;
+                        }
+
+                        let specialMentions = [];
+                        if (!pretext.startsWith('/msg')) {
+                            specialMentions = ['here', 'channel', 'all'].filter((item) => {
+                                return item.startsWith(prefix);
+                            }).map((name) => {
+                                return {username: name, type: Constants.MENTION_SPECIAL};
+                            });
+                        }
+
+                        const users = members.concat(specialMentions).concat(nonmembers);
+                        const mentions = users.map((user) => '@' + user.username);
+
+                        AppDispatcher.handleServerAction({
+                            type: ActionTypes.SUGGESTION_RECEIVED_SUGGESTIONS,
+                            id: suggestionId,
+                            matchedPretext: captured[0],
+                            terms: mentions,
+                            items: users,
+                            component: AtMentionSuggestion
+                        });
                     }
-                }
-            }
-
-            // Filter users by prefix.
-            const filteredMembers = filterUsersByPrefix(
-                    channelMembers, prefix, MaxUserSuggestions, Constants.MENTION_MEMBERS);
-            const filteredNonmembers = filterUsersByPrefix(
-                    channelNonmembers, prefix, MaxUserSuggestions - filteredMembers.length, Constants.MENTION_NONMEMBERS);
-            let filteredSpecialMentions = [];
-            if (!pretext.startsWith('/msg')) {
-                filteredSpecialMentions = ['here', 'channel', 'all'].filter((item) => {
-                    return item.startsWith(prefix);
-                }).map((name) => {
-                    return {username: name, type: Constants.MENTION_SPECIAL};
-                });
+                );
             }
 
-            // Sort users by username.
-            [filteredMembers, filteredNonmembers].forEach((items) => {
-                items.sort((a, b) => {
-                    const aPrefix = a.username.startsWith(prefix);
-                    const bPrefix = b.username.startsWith(prefix);
-
-                    if (aPrefix === bPrefix) {
-                        return a.username.localeCompare(b.username);
-                    } else if (aPrefix) {
-                        return -1;
-                    }
-
-                    return 1;
-                });
-            });
-
-            const filtered = filteredMembers.concat(filteredSpecialMentions).concat(filteredNonmembers);
-
-            const mentions = filtered.map((user) => '@' + user.username);
-
-            SuggestionStore.addSuggestions(suggestionId, mentions, filtered, AtMentionSuggestion, captured[0]);
+            this.timeoutId = setTimeout(
+                autocomplete.bind(this),
+                Constants.AUTOCOMPLETE_TIMEOUT
+            );
         }
     }
 }
diff --git a/webapp/components/suggestion/search_user_provider.jsx b/webapp/components/suggestion/search_user_provider.jsx
index b5466cf39..baf91cd94 100644
--- a/webapp/components/suggestion/search_user_provider.jsx
+++ b/webapp/components/suggestion/search_user_provider.jsx
@@ -1,13 +1,16 @@
 // Copyright (c) 2015 Mattermost, Inc. All Rights Reserved.
 // See License.txt for license information.
 
-import React from 'react';
+import Suggestion from './suggestion.jsx';
 
+import {autocompleteUsersInTeam} from 'actions/user_actions.jsx';
+
+import AppDispatcher from 'dispatcher/app_dispatcher.jsx';
 import Client from 'client/web_client.jsx';
-import SuggestionStore from 'stores/suggestion_store.jsx';
-import UserStore from 'stores/user_store.jsx';
+import * as Utils from 'utils/utils.jsx';
+import {Constants, ActionTypes} from 'utils/constants.jsx';
 
-import Suggestion from './suggestion.jsx';
+import React from 'react';
 
 class SearchUserSuggestion extends Suggestion {
     render() {
@@ -18,6 +21,17 @@ class SearchUserSuggestion extends Suggestion {
             className += ' selected';
         }
 
+        const username = item.username;
+        let description = '';
+
+        if ((item.first_name || item.last_name) && item.nickname) {
+            description = `- ${Utils.getFullName(item)} (${item.nickname})`;
+        } else if (item.nickname) {
+            description = `- (${item.nickname})`;
+        } else if (item.first_name || item.last_name) {
+            description = `- ${Utils.getFullName(item)}`;
+        }
+
         return (
             <div
                 className={className}
@@ -27,34 +41,60 @@ class SearchUserSuggestion extends Suggestion {
                     className='profile-img rounded'
                     src={Client.getUsersRoute() + '/' + item.id + '/image?time=' + item.update_at}
                 />
-                <i className='fa fa fa-plus-square'/>{item.username}
+                <i className='fa fa fa-plus-square'/>
+                <div className='mention--align'>
+                    <span>
+                        {username}
+                    </span>
+                    <span className='mention__fullname'>
+                        {' '}
+                        {description}
+                    </span>
+                </div>
             </div>
         );
     }
 }
 
 export default class SearchUserProvider {
+    constructor() {
+        this.timeoutId = '';
+    }
+
+    componentWillUnmount() {
+        clearTimeout(this.timeoutId);
+    }
+
     handlePretextChanged(suggestionId, pretext) {
+        clearTimeout(this.timeoutId);
+
         const captured = (/\bfrom:\s*(\S*)$/i).exec(pretext.toLowerCase());
         if (captured) {
             const usernamePrefix = captured[1];
 
-            const users = UserStore.getProfiles();
-            let filtered = [];
-
-            for (const id of Object.keys(users)) {
-                const user = users[id];
+            function autocomplete() {
+                autocompleteUsersInTeam(
+                    usernamePrefix,
+                    (data) => {
+                        const users = data.in_team;
+                        const mentions = users.map((user) => user.username);
 
-                if (user.username.startsWith(usernamePrefix)) {
-                    filtered.push(user);
-                }
+                        AppDispatcher.handleServerAction({
+                            type: ActionTypes.SUGGESTION_RECEIVED_SUGGESTIONS,
+                            id: suggestionId,
+                            matchedPretext: usernamePrefix,
+                            terms: mentions,
+                            items: users,
+                            component: SearchUserSuggestion
+                        });
+                    }
+                );
             }
 
-            filtered = filtered.sort((a, b) => a.username.localeCompare(b.username));
-
-            const usernames = filtered.map((user) => user.username);
-
-            SuggestionStore.addSuggestions(suggestionId, usernames, filtered, SearchUserSuggestion, usernamePrefix);
+            this.timeoutId = setTimeout(
+                autocomplete.bind(this),
+                Constants.AUTOCOMPLETE_TIMEOUT
+            );
         }
     }
 }
diff --git a/webapp/components/suggestion/suggestion_list.jsx b/webapp/components/suggestion/suggestion_list.jsx
index 7d8059e1e..65311a582 100644
--- a/webapp/components/suggestion/suggestion_list.jsx
+++ b/webapp/components/suggestion/suggestion_list.jsx
@@ -163,4 +163,4 @@ SuggestionList.propTypes = {
 
 SuggestionList.defaultProps = {
     renderDividers: false
-};
\ No newline at end of file
+};
diff --git a/webapp/components/suggestion/switch_channel_provider.jsx b/webapp/components/suggestion/switch_channel_provider.jsx
index 70e95b9b1..94622b536 100644
--- a/webapp/components/suggestion/switch_channel_provider.jsx
+++ b/webapp/components/suggestion/switch_channel_provider.jsx
@@ -4,7 +4,6 @@
 import React from 'react';
 
 import ChannelStore from 'stores/channel_store.jsx';
-import UserStore from 'stores/user_store.jsx';
 import SuggestionStore from 'stores/suggestion_store.jsx';
 import Suggestion from './suggestion.jsx';
 import Constants from 'utils/constants.jsx';
@@ -58,7 +57,10 @@ export default class SwitchChannelProvider {
                 const channel = allChannels[id];
                 if (channel.display_name.toLowerCase().startsWith(channelPrefix.toLowerCase())) {
                     channels.push(channel);
-                } else if (channel.type === Constants.DM_CHANNEL && Utils.getDirectTeammate(channel.id).username.startsWith(channelPrefix.toLowerCase())) {
+                }
+
+                // TODO: Fix with  auto-complete refactor
+                /*else if (channel.type === Constants.DM_CHANNEL && Utils.getDirectTeammate(channel.id).username.startsWith(channelPrefix.toLowerCase())) {
                     // New channel to not modify existing channel
                     const otherUser = Utils.getDirectTeammate(channel.id);
                     const newChannel = {
@@ -68,7 +70,7 @@ export default class SwitchChannelProvider {
                         status: UserStore.getStatus(otherUser.id) || 'offline'
                     };
                     channels.push(newChannel);
-                }
+                }*/
             }
 
             channels.sort((a, b) => {