Merging performance branch into master (#4268)

* improve performance on sendNotifications

* Fix SQL queries

* Remove get direct profiles, not needed anymore

* Add raw data to error details if AppError fails to decode

* men

* Fix decode (#4052)

* Fixing json decode

* Adding unit test

* Initial work for client scaling (#4051)

* Begin adding paging to profiles API

* Added more paging functionality

* Finish hooking up admin console user lists

* Add API for searching users and add searching to all user lists

* Add lazy loading of profiles

* Revert config.json

* Fix unit tests and some style issues

* Add GetProfilesFromList to Go driver and fix web unit test

* Update etag for GetProfiles

* Updating ui for filters and pagination (#4044)

* Updating UI for pagination

* Adjusting margins for filter row

* Adjusting margin for specific modals

* Adding relative padding to system console

* Adjusting responsive view

* Update client user tests

* Minor fixes for direct messages modal (#4056)

* Remove some unneeded initial load calls (#4057)

* UX updates to user lists, added smart counts and bug fixes (#4059)

* Improved getExplicitMentions and unit tests (#4064)

* Refactor getting posts to lazy load profiles correctly (#4062)

* Comment out SetActiveChannel test (#4066)

* Profiler cpu, block, and memory profiler. (#4081)

* Fix TestSetActiveChannel unit test (#4071)

* Fixing build failure caused by dependancies updating (#4076)

* Adding profiler

* Fix admin_team_member_dropdown eslint errors

* Bumping session cache size (#4077)

* Bumping session cache size

* Bumping status cache

* Refactor how the client handles channel members to be large team friendly (#4106)

* Refactor how the client handles channel members to be large team friendly

* Change Id to ChannelId in ChannelStats model

* Updated getChannelMember and getProfilesByIds routes to match proposal

* Performance improvements (#4100)

* Performance improvements

* Fixing re-connect issue

* Fixing error message

* Some other minor perf tweaks

* Some other minor perf tweaks

* Fixing config file

* Fixing buffer size

* Fixing web socket send message

* adding some error logging

* fix getMe to be user required

* Fix websocket event for new user

* Fixing shutting down

* Reverting web socket changes

* Fixing logging lvl

* Adding caching to GetMember

* Adding some logging

* Fixing caching

* Fixing caching invalidate

* Fixing direct message caching

* Fixing caching

* Fixing caching

* Remove GetDirectProfiles from initial load

* Adding logging and fixing websocket client

* Adding back caching from bad merge.

* Explicitly close go driver requests (#4162)

* Refactored how the client handles team members to be more large team friendly (#4159)

* Refactor getProfilesForDirectMessageList API into getAllProfiles API

* Refactored how the client handles team members to be more large team friendly

* Fix js error when receiving a notification

* Fix JS error caused by current user being overwritten with sanitized version (#4165)

* Adding error message to status failure (#4167)

* Fix a few bugs caused by client scaling refactoring (#4170)

* When there is no read replica, don't open a second set of connections to the master database (#4173)

* Adding connection tacking to stats (#4174)

* Reduce DB writes for statuses and other status related changes (#4175)

* Fix bug preventing opening of DM channels from more modal (#4181)

* 	Fixing socket timing error (#4183)

* Fixing ping/pong handler

* Fixing socket timing error

* Commenting out status broadcasting

* Removing user status changes

* Removing user status changes

* Removing user status changes

* Removing user status changes

* Adding DoPreComputeJson()

* Performance improvements (#4194)

* * Fix System Console Analytics queries
* Add db.SetConnMaxLifetime to 15 minutes
* Add "net/http/pprof" for profiling
* Add FreeOSMemory() to manually release memory on reload config

* Add flag to enable http profiler

* Fix memory leak (#4197)

* Fix memory leak

* removed unneeded nil assignment

* Fixing go routine leak (#4208)

* Merge fixes

* Merge fix

* Refactored statuses to be queried by the client rather than broadcast by the server (#4212)

* Refactored server code to reduce status broadcasts and to allow getting statuses by IDs

* Refactor client code to periodically fetch statuses

* Add store unit test for getting statuses by ids

* Fix status unit test

* Add getStatusesByIds REST API and move the client over to use that instead of the WebSocket

* Adding multiple threads to websocket hub (#4230)

* Adding multiple threads to websocket hub

* Fixing unit tests

* Fixing so websocket connections from the same user end up in the same… (#4240)

* Fixing so websocket connections from the same user end up in the same list

* Removing old comment

* Refactor user autocomplete to query the server (#4239)

* Add API for autocompleting users

* Converted at mention autocomplete to query server

* Converted user search autocomplete to query server

* Switch autocomplete API naming to use term instead of username

* Split autocomplete API into two, one for channels and for teams

* Fix copy/paste error

* Some final client scaling fixes (#4246)

* Add lazy loading of profiles to integration pages

* Add lazy loading of profiles to emoji page

* Fix JS error when receiving post in select team menu and also clean up channel store

1 files changed, 140 insertions, 61 deletions

diff --git a/webapp/components/more_direct_channels.jsx b/webapp/components/more_direct_channels.jsx
index 24718387e..11849f718 100644
--- a/webapp/components/more_direct_channels.jsx
+++ b/webapp/components/more_direct_channels.jsx
@@ -1,73 +1,67 @@
 // Copyright (c) 2015 Mattermost, Inc. All Rights Reserved.
 // See License.txt for license information.
 
-import FilteredUserList from 'components/filtered_user_list.jsx';
+import SearchableUserList from 'components/searchable_user_list.jsx';
 import SpinnerButton from 'components/spinner_button.jsx';
-import LoadingScreen from 'components/loading_screen.jsx';
 
-import {getMoreDmList} from 'actions/user_actions.jsx';
+import {searchUsers} from 'actions/user_actions.jsx';
+import {openDirectChannelToUser} from 'actions/channel_actions.jsx';
 
 import UserStore from 'stores/user_store.jsx';
 import TeamStore from 'stores/team_store.jsx';
 
+import * as AsyncClient from 'utils/async_client.jsx';
 import * as Utils from 'utils/utils.jsx';
+import Constants from 'utils/constants.jsx';
 
 import React from 'react';
 import {Modal} from 'react-bootstrap';
 import {FormattedMessage} from 'react-intl';
 import {browserHistory} from 'react-router/es6';
 
+const USERS_PER_PAGE = 50;
+
 export default class MoreDirectChannels extends React.Component {
     constructor(props) {
         super(props);
 
         this.handleHide = this.handleHide.bind(this);
         this.handleShowDirectChannel = this.handleShowDirectChannel.bind(this);
-        this.handleUserChange = this.handleUserChange.bind(this);
-        this.onTeamChange = this.onTeamChange.bind(this);
+        this.onChange = this.onChange.bind(this);
         this.createJoinDirectChannelButton = this.createJoinDirectChannelButton.bind(this);
+        this.toggleList = this.toggleList.bind(this);
+        this.nextPage = this.nextPage.bind(this);
+        this.search = this.search.bind(this);
+        this.loadComplete = this.loadComplete.bind(this);
 
         this.state = {
-            users: UserStore.getProfilesForDmList(),
-            teamMembers: TeamStore.getMembersForTeam(),
+            users: UserStore.getProfileListInTeam(TeamStore.getCurrentId(), true),
             loadingDMChannel: -1,
-            usersLoaded: false,
-            teamMembersLoaded: false
+            listType: 'team',
+            loading: false,
+            search: false
         };
     }
 
     componentDidMount() {
-        UserStore.addDmListChangeListener(this.handleUserChange);
-        TeamStore.addChangeListener(this.onTeamChange);
+        UserStore.addChangeListener(this.onChange);
+        UserStore.addInTeamChangeListener(this.onChange);
+        UserStore.addStatusesChangeListener(this.onChange);
+        TeamStore.addChangeListener(this.onChange);
+
+        AsyncClient.getProfiles(0, Constants.PROFILE_CHUNK_SIZE);
+        AsyncClient.getProfilesInTeam(TeamStore.getCurrentId(), 0, Constants.PROFILE_CHUNK_SIZE);
     }
 
     componentWillUnmount() {
-        UserStore.removeDmListChangeListener(this.handleUserChange);
-        TeamStore.removeChangeListener(this.onTeamChange);
+        UserStore.removeChangeListener(this.onChange);
+        UserStore.removeInTeamChangeListener(this.onChange);
+        UserStore.removeStatusesChangeListener(this.onChange);
+        TeamStore.removeChangeListener(this.onChange);
     }
 
-    shouldComponentUpdate(nextProps, nextState) {
-        if (nextProps.show !== this.props.show) {
-            return true;
-        }
-
-        if (nextProps.onModalDismissed.toString() !== this.props.onModalDismissed.toString()) {
-            return true;
-        }
-
-        if (nextState.loadingDMChannel !== this.state.loadingDMChannel) {
-            return true;
-        }
-
-        if (!Utils.areObjectsEqual(nextState.users, this.state.users)) {
-            return true;
-        }
-
-        if (!Utils.areObjectsEqual(nextState.teamMembers, this.state.teamMembers)) {
-            return true;
-        }
-
-        return false;
+    loadComplete() {
+        this.setState({loading: false});
     }
 
     handleHide() {
@@ -84,7 +78,7 @@ export default class MoreDirectChannels extends React.Component {
         }
 
         this.setState({loadingDMChannel: teammate.id});
-        Utils.openDirectChannelToUser(
+        openDirectChannelToUser(
             teammate,
             (channel) => {
                 browserHistory.push(TeamStore.getCurrentTeamUrl() + '/channels/' + channel.name);
@@ -97,17 +91,35 @@ export default class MoreDirectChannels extends React.Component {
         );
     }
 
-    handleUserChange() {
+    onChange(force) {
+        if (this.state.search && !force) {
+            return;
+        }
+
+        let users;
+        if (this.state.listType === 'any') {
+            users = UserStore.getProfileList();
+        } else {
+            users = UserStore.getProfileListInTeam(TeamStore.getCurrentId(), true);
+        }
+
         this.setState({
-            users: UserStore.getProfilesForDmList(),
-            usersLoaded: true
+            users
         });
     }
 
-    onTeamChange() {
+    toggleList(e) {
+        const listType = e.target.value;
+        let users;
+        if (listType === 'any') {
+            users = UserStore.getProfileList();
+        } else {
+            users = UserStore.getProfileListInTeam(TeamStore.getCurrentId(), true);
+        }
+
         this.setState({
-            teamMembers: TeamStore.getMembersForTeam(),
-            teamMembersLoaded: true
+            users,
+            listType
         });
     }
 
@@ -126,38 +138,96 @@ export default class MoreDirectChannels extends React.Component {
         );
     }
 
+    nextPage(page) {
+        if (this.state.listType === 'any') {
+            AsyncClient.getProfiles((page + 1) * USERS_PER_PAGE, USERS_PER_PAGE);
+        } else {
+            AsyncClient.getProfilesInTeam(TeamStore.getCurrentId(), (page + 1) * USERS_PER_PAGE, USERS_PER_PAGE);
+        }
+    }
+
+    search(term) {
+        if (term === '') {
+            this.onChange(true);
+            this.setState({search: false});
+            return;
+        }
+
+        let teamId;
+        if (this.state.listType === 'any') {
+            teamId = '';
+        } else {
+            teamId = TeamStore.getCurrentId();
+        }
+
+        searchUsers(
+            term,
+            teamId,
+            {},
+            (users) => {
+                for (let i = 0; i < users.length; i++) {
+                    if (users[i].id === UserStore.getCurrentId()) {
+                        users.splice(i, 1);
+                        break;
+                    }
+                }
+                this.setState({search: true, users});
+            }
+        );
+    }
+
     render() {
         let maxHeight = 1000;
         if (Utils.windowHeight() <= 1200) {
             maxHeight = Utils.windowHeight() - 300;
         }
 
-        var body = null;
-        if (!this.state.usersLoaded || !this.state.teamMembersLoaded) {
-            body = (<LoadingScreen/>);
-        } else {
-            var showTeamToggle = false;
-            if (global.window.mm_config.RestrictDirectMessage === 'any') {
-                showTeamToggle = true;
-            }
-
-            body = (
-                <FilteredUserList
-                    style={{maxHeight}}
-                    users={this.state.users}
-                    teamMembers={this.state.teamMembers}
-                    actions={[this.createJoinDirectChannelButton]}
-                    showTeamToggle={showTeamToggle}
-                />
+        let teamToggle;
+        if (global.window.mm_config.RestrictDirectMessage === 'any') {
+            teamToggle = (
+                <div className='member-select__container'>
+                    <select
+                        className='form-control'
+                        id='restrictList'
+                        ref='restrictList'
+                        defaultValue='team'
+                        onChange={this.toggleList}
+                    >
+                        <option value='any'>
+                            <FormattedMessage
+                                id='filtered_user_list.any_team'
+                                defaultMessage='All Users'
+                            />
+                        </option>
+                        <option value='team'>
+                            <FormattedMessage
+                                id='filtered_user_list.team_only'
+                                defaultMessage='Members of this Team'
+                            />
+                        </option>
+                    </select>
+                    <span
+                        className='member-show'
+                    >
+                        <FormattedMessage
+                            id='filtered_user_list.show'
+                            defaultMessage='Filter:'
+                        />
+                    </span>
+                </div>
             );
         }
 
+        let users = this.state.users;
+        if (this.state.loading) {
+            users = null;
+        }
+
         return (
             <Modal
                 dialogClassName='more-modal more-direct-channels'
                 show={this.props.show}
                 onHide={this.handleHide}
-                onEntered={getMoreDmList}
             >
                 <Modal.Header closeButton={true}>
                     <Modal.Title>
@@ -168,7 +238,16 @@ export default class MoreDirectChannels extends React.Component {
                     </Modal.Title>
                 </Modal.Header>
                 <Modal.Body>
-                    {body}
+                    {teamToggle}
+                    <SearchableUserList
+                        key={'moreDirectChannelsList_' + this.state.listType}
+                        style={{maxHeight}}
+                        users={users}
+                        usersPerPage={USERS_PER_PAGE}
+                        nextPage={this.nextPage}
+                        search={this.search}
+                        actions={[this.createJoinDirectChannelButton]}
+                    />
                 </Modal.Body>
                 <Modal.Footer>
                     <button